2024-02-07T12:08:56.887 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-07T12:08:56.887 DEBUG:teuthology.run:Teuthology command: teuthology --name gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi --owner scheduled_gabrioux@teuthology --verbose --archive /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731 --description orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} -- /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/orig.config.yaml 2024-02-07T12:08:56.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-07T12:08:56.998 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731 branch: wip-guits-testing-4-2024-02-05-1854 description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} email: gabrioux@ibm.com first_in_suite: false job_id: '7548731' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-4-2024-02-05-1854 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-4-2024-02-05-1854 sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 owner: scheduled_gabrioux@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 1701 sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-4-2024-02-05-1854 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_44458ccedb18241cc2e17f10cdab3d1385399261/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 targets: smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGGzjAw/GvhXBhlnFu+gD8gFQGNZF//ng8zqgP/wRhJX9aRgUCboL3ELhBsXvRvRVT6mYSztKWOAr7qM6lV1WI= smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgg54BK6U3EbSSrDWNc7km5nWVlgFlbxYqu41t1/gKZUKN1a+B4LvjqNy0fd24sodoJ4JVAbQadKyA6UJjEO7M= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNtRBEv0PJalB1W6PuKp2BrtE85xVRvqBzFrRTI93cIBXDj7UweVgYon6xubQqvB7sVtICOhWgGsQITHs2O0Nnk= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-06_08:19:47 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-07T12:08:56.999 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_44458ccedb18241cc2e17f10cdab3d1385399261/qa; will attempt to use it 2024-02-07T12:08:56.999 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_44458ccedb18241cc2e17f10cdab3d1385399261/qa/tasks 2024-02-07T12:08:56.999 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-07T12:08:57.001 INFO:teuthology.task.internal:Checking packages... 2024-02-07T12:08:57.026 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '44458ccedb18241cc2e17f10cdab3d1385399261' 2024-02-07T12:08:57.026 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-07T12:08:57.026 INFO:teuthology.packaging:ref: None 2024-02-07T12:08:57.026 INFO:teuthology.packaging:tag: None 2024-02-07T12:08:57.027 INFO:teuthology.packaging:branch: wip-guits-testing-4-2024-02-05-1854 2024-02-07T12:08:57.027 INFO:teuthology.packaging:sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T12:08:57.027 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-4-2024-02-05-1854 2024-02-07T12:08:57.166 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1252.g44458cce 2024-02-07T12:08:57.168 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-07T12:08:57.175 INFO:teuthology.task.internal:no buildpackages task found 2024-02-07T12:08:57.175 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-07T12:08:57.193 INFO:teuthology.task.internal:Saving configuration 2024-02-07T12:08:57.203 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-07T12:08:57.209 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T12:08:57.235 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:03:51.514475', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGGzjAw/GvhXBhlnFu+gD8gFQGNZF//ng8zqgP/wRhJX9aRgUCboL3ELhBsXvRvRVT6mYSztKWOAr7qM6lV1WI='} 2024-02-07T12:08:57.260 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:03:51.515692', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgg54BK6U3EbSSrDWNc7km5nWVlgFlbxYqu41t1/gKZUKN1a+B4LvjqNy0fd24sodoJ4JVAbQadKyA6UJjEO7M='} 2024-02-07T12:08:57.283 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:03:51.516664', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNtRBEv0PJalB1W6PuKp2BrtE85xVRvqBzFrRTI93cIBXDj7UweVgYon6xubQqvB7sVtICOhWgGsQITHs2O0Nnk='} 2024-02-07T12:08:57.283 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-07T12:08:57.290 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-07T12:08:57.290 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-02-07T12:08:57.290 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-02-07T12:08:57.290 INFO:teuthology.run_tasks:Running task console_log... 2024-02-07T12:08:57.399 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f13dcc18f70>, signals=[15]) 2024-02-07T12:08:57.400 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-07T12:08:57.405 INFO:teuthology.task.internal:Opening connections... 2024-02-07T12:08:57.405 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-02-07T12:08:57.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:08:57.513 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-02-07T12:08:57.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:08:57.604 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-02-07T12:08:57.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:08:57.695 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-07T12:08:57.702 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-02-07T12:08:57.719 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-02-07T12:08:57.720 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-02-07T12:08:57.775 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-07T12:08:57.776 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-02-07T12:08:57.777 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-07T12:08:57.777 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-07T12:08:57.777 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-07T12:08:57.777 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-02-07T12:08:57.803 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-02-07T12:08:57.822 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-02-07T12:08:57.822 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-02-07T12:08:57.880 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-07T12:08:57.881 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-07T12:08:57.882 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-07T12:08:57.882 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-02-07T12:08:57.903 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-02-07T12:08:57.920 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-02-07T12:08:57.921 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-02-07T12:08:57.977 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-07T12:08:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-07T12:08:57.979 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-02-07T12:08:57.994 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-07T12:08:58.003 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-07T12:08:58.019 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-07T12:08:58.019 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-02-07T12:08:58.022 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-02-07T12:08:58.024 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-02-07T12:08:58.039 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-07T12:08:58.045 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-07T12:08:58.045 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-02-07T12:08:58.079 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-02-07T12:08:58.082 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-02-07T12:08:58.343 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-07T12:08:58.423 INFO:teuthology.run_tasks:Running task kernel... 2024-02-07T12:08:58.438 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-07T12:08:58.438 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-07T12:08:58.438 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-07T12:08:58.439 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-07T12:08:58.439 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-07T12:08:58.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:08:58.456 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-02-07T12:08:58.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:08:58.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:08:58.458 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-02-07T12:08:58.458 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-02-07T12:08:58.513 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-361.el9.x86_64 2024-02-07T12:08:58.513 INFO:teuthology.task.kernel:Running kernel on smithi078: 5.14.0-361.el9.x86_64 2024-02-07T12:08:58.513 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-02-07T12:08:58.516 INFO:teuthology.orchestra.run.smithi106.stdout:5.14.0-361.el9.x86_64 2024-02-07T12:08:58.516 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-361.el9.x86_64 2024-02-07T12:08:58.516 INFO:teuthology.task.kernel:Running kernel on smithi106: 5.14.0-361.el9.x86_64 2024-02-07T12:08:58.517 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum install -y kernel 2024-02-07T12:08:58.517 INFO:teuthology.task.kernel:Running kernel on smithi171: 5.14.0-361.el9.x86_64 2024-02-07T12:08:58.517 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum install -y kernel 2024-02-07T12:09:00.634 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-02-07T12:09:00.757 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-02-07T12:09:00.777 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-02-07T12:09:02.198 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-02-07T12:09:02.276 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-02-07T12:09:02.361 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2024-02-07T12:09:03.601 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.0 MB 00:00 2024-02-07T12:09:03.881 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.0 MB 00:01 2024-02-07T12:09:06.018 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-02-07T12:09:06.378 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-02-07T12:09:10.127 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 5.9 MB 00:00 2024-02-07T12:09:10.514 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 5.9 MB 00:00 2024-02-07T12:09:11.901 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 15 kB 00:00 2024-02-07T12:09:12.004 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-02-07T12:09:13.668 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-02-07T12:09:18.588 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-07T12:09:20.754 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-07T12:09:20.754 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-07T12:09:20.875 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-02-07T12:09:20.884 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-02-07T12:09:20.885 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-02-07T12:09:20.885 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-02-07T12:09:20.885 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-02-07T12:09:20.885 INFO:teuthology.orchestra.run.smithi171.stdout: kernel x86_64 5.14.0-418.el9 CentOS-BaseOS 6.0 M 2024-02-07T12:09:20.885 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-02-07T12:09:20.886 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core x86_64 5.14.0-418.el9 CentOS-BaseOS 21 M 2024-02-07T12:09:20.886 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules x86_64 5.14.0-418.el9 CentOS-BaseOS 39 M 2024-02-07T12:09:20.886 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core x86_64 5.14.0-418.el9 CentOS-BaseOS 33 M 2024-02-07T12:09:20.886 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:09:20.886 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-02-07T12:09:20.886 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-02-07T12:09:20.886 INFO:teuthology.orchestra.run.smithi171.stdout:Install 4 Packages 2024-02-07T12:09:20.886 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:09:20.887 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 99 M 2024-02-07T12:09:20.887 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 122 M 2024-02-07T12:09:20.887 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-02-07T12:09:21.448 INFO:teuthology.orchestra.run.smithi171.stdout:(1/4): kernel-5.14.0-418.el9.x86_64.rpm 11 MB/s | 6.0 MB 00:00 2024-02-07T12:09:22.340 INFO:teuthology.orchestra.run.smithi171.stdout:(2/4): kernel-core-5.14.0-418.el9.x86_64.rpm 14 MB/s | 21 MB 00:01 2024-02-07T12:09:23.540 INFO:teuthology.orchestra.run.smithi171.stdout:(3/4): kernel-modules-core-5.14.0-418.el9.x86_6 16 MB/s | 33 MB 00:02 2024-02-07T12:09:23.882 INFO:teuthology.orchestra.run.smithi171.stdout:(4/4): kernel-modules-5.14.0-418.el9.x86_64.rpm 13 MB/s | 39 MB 00:02 2024-02-07T12:09:23.885 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:09:23.885 INFO:teuthology.orchestra.run.smithi171.stdout:Total 33 MB/s | 99 MB 00:02 2024-02-07T12:09:23.886 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-02-07T12:09:24.280 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-02-07T12:09:24.280 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-02-07T12:09:25.349 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-02-07T12:09:25.350 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-02-07T12:09:27.814 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-02-07T12:09:28.589 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-core-5.14.0-418.el9.x86_64 1/4 2024-02-07T12:09:28.726 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T12:09:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T12:09:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T12:09:35.794 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 875 kB/s | 20 MB 00:23 2024-02-07T12:09:36.409 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T12:09:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:09:40.866 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 5.1 kB/s | 1.7 kB 00:00 2024-02-07T12:09:41.264 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:09:43.015 INFO:teuthology.orchestra.run.smithi106.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-07T12:09:43.015 INFO:teuthology.orchestra.run.smithi106.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-07T12:09:43.118 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout: kernel x86_64 5.14.0-418.el9 CentOS-BaseOS 6.0 M 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-core x86_64 5.14.0-418.el9 CentOS-BaseOS 21 M 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-modules x86_64 5.14.0-418.el9 CentOS-BaseOS 39 M 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-modules-core x86_64 5.14.0-418.el9 CentOS-BaseOS 33 M 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:09:43.130 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-02-07T12:09:43.131 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-02-07T12:09:43.131 INFO:teuthology.orchestra.run.smithi106.stdout:Install 4 Packages 2024-02-07T12:09:43.131 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:09:43.131 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 99 M 2024-02-07T12:09:43.131 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 122 M 2024-02-07T12:09:43.131 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-02-07T12:09:43.959 INFO:teuthology.orchestra.run.smithi106.stdout:(1/4): kernel-5.14.0-418.el9.x86_64.rpm 7.2 MB/s | 6.0 MB 00:00 2024-02-07T12:09:44.426 INFO:teuthology.orchestra.run.smithi106.stdout:(2/4): kernel-core-5.14.0-418.el9.x86_64.rpm 16 MB/s | 21 MB 00:01 2024-02-07T12:09:45.268 INFO:teuthology.orchestra.run.smithi106.stdout:(3/4): kernel-modules-5.14.0-418.el9.x86_64.rpm 18 MB/s | 39 MB 00:02 2024-02-07T12:09:46.018 INFO:teuthology.orchestra.run.smithi106.stdout:(4/4): kernel-modules-core-5.14.0-418.el9.x86_6 16 MB/s | 33 MB 00:02 2024-02-07T12:09:46.021 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:09:46.021 INFO:teuthology.orchestra.run.smithi106.stdout:Total 34 MB/s | 99 MB 00:02 2024-02-07T12:09:46.021 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-02-07T12:09:46.383 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-02-07T12:09:46.383 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-02-07T12:09:47.413 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-02-07T12:09:47.413 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-02-07T12:09:49.811 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-02-07T12:09:50.547 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : kernel-modules-core-5.14.0-418.el9.x86_64 1/4 2024-02-07T12:09:50.685 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T12:09:53.462 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T12:09:53.647 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T12:09:58.342 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T12:09:58.450 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:02.760 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 136 kB/s | 8.0 MB 01:00 2024-02-07T12:10:03.149 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:05.655 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-02-07T12:10:10.324 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:10.324 INFO:teuthology.orchestra.run.smithi171.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-07T12:10:10.324 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:10:10.395 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-5.14.0-418.el9.x86_64 1/4 2024-02-07T12:10:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T12:10:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T12:10:10.995 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.9 MB 00:01 2024-02-07T12:10:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:10:11.254 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-02-07T12:10:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-5.14.0-418.el9.x86_64 2024-02-07T12:10:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core-5.14.0-418.el9.x86_64 2024-02-07T12:10:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-5.14.0-418.el9.x86_64 2024-02-07T12:10:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core-5.14.0-418.el9.x86_64 2024-02-07T12:10:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:10:11.255 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-02-07T12:10:11.764 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:10:11.895 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T12:10:11.896 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T12:10:11.896 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-418.el9.x86_64 2024-02-07T12:10:11.896 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-07T12:10:11.896 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-07T12:10:11.896 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-02-07T12:10:11.896 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi171.front.sepia.ceph.com, path=None, version=distro) 2024-02-07T12:10:11.897 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:10:12.005 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T12:10:12.005 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T12:10:12.005 DEBUG:teuthology.orchestra.run.smithi171:> sudo rpm -qi grub2-tools 2024-02-07T12:10:12.041 INFO:teuthology.orchestra.run.smithi171.stdout:Name : grub2-tools 2024-02-07T12:10:12.041 INFO:teuthology.orchestra.run.smithi171.stdout:Epoch : 1 2024-02-07T12:10:12.041 INFO:teuthology.orchestra.run.smithi171.stdout:Version : 2.06 2024-02-07T12:10:12.041 INFO:teuthology.orchestra.run.smithi171.stdout:Release : 68.el9 2024-02-07T12:10:12.041 INFO:teuthology.orchestra.run.smithi171.stdout:Architecture: x86_64 2024-02-07T12:10:12.041 INFO:teuthology.orchestra.run.smithi171.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Group : Unspecified 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Size : 8264254 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:License : GPLv3+ 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Packager : builder@centos.org 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Vendor : CentOS 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Summary : Support tools for GRUB. 2024-02-07T12:10:12.042 INFO:teuthology.orchestra.run.smithi171.stdout:Description : 2024-02-07T12:10:12.043 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:10:12.043 INFO:teuthology.orchestra.run.smithi171.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-07T12:10:12.043 INFO:teuthology.orchestra.run.smithi171.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-07T12:10:12.043 INFO:teuthology.orchestra.run.smithi171.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-07T12:10:12.043 INFO:teuthology.orchestra.run.smithi171.stdout:hardware devices. 2024-02-07T12:10:12.043 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:10:12.043 INFO:teuthology.orchestra.run.smithi171.stdout:This subpackage provides tools for support of all platforms. 2024-02-07T12:10:12.043 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-07T12:10:12.043 INFO:teuthology.task.kernel:Updating grub on smithi171 to boot 5.14.0-418.el9.x86_64 2024-02-07T12:10:12.043 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-07T12:10:12.496 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 15 kB 00:00 2024-02-07T12:10:12.660 INFO:teuthology.orchestra.run.smithi171.stderr:Generating grub configuration file ... 2024-02-07T12:10:13.615 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:00 2024-02-07T12:10:14.008 INFO:teuthology.orchestra.run.smithi171.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-07T12:10:14.051 INFO:teuthology.orchestra.run.smithi171.stderr:done 2024-02-07T12:10:14.054 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-02-07T12:10:14.069 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.otaq3ZzR0N 2024-02-07T12:10:14.070 DEBUG:teuthology.orchestra.run.smithi171:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.otaq3ZzR0N 2024-02-07T12:10:14.138 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.otaq3ZzR0N 2024-02-07T12:10:14.269 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.otaq3ZzR0N is 6KB 2024-02-07T12:10:14.291 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.otaq3ZzR0N 2024-02-07T12:10:14.306 DEBUG:teuthology.orchestra.run.smithi171:> sudo /bin/ls /boot/loader/entries || true 2024-02-07T12:10:14.370 INFO:teuthology.orchestra.run.smithi171.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-07T12:10:14.370 INFO:teuthology.orchestra.run.smithi171.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-07T12:10:14.370 INFO:teuthology.orchestra.run.smithi171.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-07T12:10:14.371 INFO:teuthology.orchestra.run.smithi171.stdout:15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64.conf 2024-02-07T12:10:14.372 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64 2024-02-07T12:10:14.509 DEBUG:teuthology.orchestra.run.smithi171:> sudo shutdown -r now 2024-02-07T12:10:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 8.1 kB/s | 1.7 kB 00:00 2024-02-07T12:10:20.721 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-07T12:10:20.721 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-07T12:10:20.818 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-418.el9 CentOS-BaseOS 6.0 M 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core x86_64 5.14.0-418.el9 CentOS-BaseOS 21 M 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules x86_64 5.14.0-418.el9 CentOS-BaseOS 39 M 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core x86_64 5.14.0-418.el9 CentOS-BaseOS 33 M 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-07T12:10:20.826 INFO:teuthology.orchestra.run.smithi078.stdout:Install 4 Packages 2024-02-07T12:10:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:10:20.827 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 99 M 2024-02-07T12:10:20.827 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 122 M 2024-02-07T12:10:20.827 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-02-07T12:10:21.158 INFO:teuthology.orchestra.run.smithi078.stdout:(1/4): kernel-5.14.0-418.el9.x86_64.rpm 18 MB/s | 6.0 MB 00:00 2024-02-07T12:10:21.650 INFO:teuthology.orchestra.run.smithi078.stdout:(2/4): kernel-core-5.14.0-418.el9.x86_64.rpm 25 MB/s | 21 MB 00:00 2024-02-07T12:10:22.316 INFO:teuthology.orchestra.run.smithi078.stdout:(3/4): kernel-modules-core-5.14.0-418.el9.x86_6 29 MB/s | 33 MB 00:01 2024-02-07T12:10:22.667 INFO:teuthology.orchestra.run.smithi078.stdout:(4/4): kernel-modules-5.14.0-418.el9.x86_64.rpm 21 MB/s | 39 MB 00:01 2024-02-07T12:10:22.670 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:10:22.670 INFO:teuthology.orchestra.run.smithi078.stdout:Total 54 MB/s | 99 MB 00:01 2024-02-07T12:10:22.670 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-02-07T12:10:23.035 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-02-07T12:10:23.035 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-02-07T12:10:24.086 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-02-07T12:10:24.086 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-02-07T12:10:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-02-07T12:10:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-core-5.14.0-418.el9.x86_64 1/4 2024-02-07T12:10:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T12:10:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T12:10:30.381 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T12:10:32.209 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:32.209 INFO:teuthology.orchestra.run.smithi106.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-07T12:10:32.209 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:10:32.278 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:32.675 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:32.675 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-5.14.0-418.el9.x86_64 1/4 2024-02-07T12:10:32.675 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T12:10:32.676 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T12:10:33.138 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:33.138 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:10:33.139 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-02-07T12:10:33.139 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-5.14.0-418.el9.x86_64 2024-02-07T12:10:33.139 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-core-5.14.0-418.el9.x86_64 2024-02-07T12:10:33.139 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-modules-5.14.0-418.el9.x86_64 2024-02-07T12:10:33.139 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-modules-core-5.14.0-418.el9.x86_64 2024-02-07T12:10:33.139 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:10:33.139 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-02-07T12:10:33.644 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:10:33.774 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T12:10:33.774 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T12:10:33.775 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-418.el9.x86_64 2024-02-07T12:10:33.775 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-07T12:10:33.775 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-07T12:10:33.775 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-07T12:10:33.775 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi106.front.sepia.ceph.com, path=None, version=distro) 2024-02-07T12:10:33.775 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:10:33.882 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T12:10:33.883 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T12:10:33.883 DEBUG:teuthology.orchestra.run.smithi106:> sudo rpm -qi grub2-tools 2024-02-07T12:10:33.926 INFO:teuthology.orchestra.run.smithi106.stdout:Name : grub2-tools 2024-02-07T12:10:33.926 INFO:teuthology.orchestra.run.smithi106.stdout:Epoch : 1 2024-02-07T12:10:33.926 INFO:teuthology.orchestra.run.smithi106.stdout:Version : 2.06 2024-02-07T12:10:33.926 INFO:teuthology.orchestra.run.smithi106.stdout:Release : 68.el9 2024-02-07T12:10:33.926 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: x86_64 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:Group : Unspecified 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:Size : 8264254 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:License : GPLv3+ 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:Packager : builder@centos.org 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:Vendor : CentOS 2024-02-07T12:10:33.927 INFO:teuthology.orchestra.run.smithi106.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-07T12:10:33.928 INFO:teuthology.orchestra.run.smithi106.stdout:Summary : Support tools for GRUB. 2024-02-07T12:10:33.928 INFO:teuthology.orchestra.run.smithi106.stdout:Description : 2024-02-07T12:10:33.928 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:10:33.928 INFO:teuthology.orchestra.run.smithi106.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-07T12:10:33.928 INFO:teuthology.orchestra.run.smithi106.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-07T12:10:33.928 INFO:teuthology.orchestra.run.smithi106.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-07T12:10:33.928 INFO:teuthology.orchestra.run.smithi106.stdout:hardware devices. 2024-02-07T12:10:33.929 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:10:33.929 INFO:teuthology.orchestra.run.smithi106.stdout:This subpackage provides tools for support of all platforms. 2024-02-07T12:10:33.930 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-07T12:10:33.930 INFO:teuthology.task.kernel:Updating grub on smithi106 to boot 5.14.0-418.el9.x86_64 2024-02-07T12:10:33.930 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-07T12:10:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2024-02-07T12:10:35.064 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T12:10:35.252 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-07T12:10:35.931 INFO:teuthology.orchestra.run.smithi106.stderr:done 2024-02-07T12:10:35.933 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2024-02-07T12:10:35.948 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.LTWfZhi2wl 2024-02-07T12:10:35.948 DEBUG:teuthology.orchestra.run.smithi106:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LTWfZhi2wl 2024-02-07T12:10:36.025 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.LTWfZhi2wl 2024-02-07T12:10:36.155 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.LTWfZhi2wl is 6KB 2024-02-07T12:10:36.203 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.LTWfZhi2wl 2024-02-07T12:10:36.218 DEBUG:teuthology.orchestra.run.smithi106:> sudo /bin/ls /boot/loader/entries || true 2024-02-07T12:10:36.283 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-07T12:10:36.283 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-07T12:10:36.283 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-07T12:10:36.283 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64.conf 2024-02-07T12:10:36.284 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64 2024-02-07T12:10:36.412 DEBUG:teuthology.orchestra.run.smithi106:> sudo shutdown -r now 2024-02-07T12:10:39.976 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:10:44.511 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-02-07T12:10:44.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-02-07T12:10:44.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:11:02.844 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-02-07T12:11:06.415 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-07T12:11:06.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-02-07T12:11:06.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:11:09.395 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:11:09.395 INFO:teuthology.orchestra.run.smithi078.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-07T12:11:09.395 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:11:09.464 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:11:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:11:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-418.el9.x86_64 1/4 2024-02-07T12:11:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T12:11:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T12:11:10.310 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T12:11:10.310 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:11:10.310 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-02-07T12:11:10.310 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-5.14.0-418.el9.x86_64 2024-02-07T12:11:10.310 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core-5.14.0-418.el9.x86_64 2024-02-07T12:11:10.311 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-5.14.0-418.el9.x86_64 2024-02-07T12:11:10.311 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core-5.14.0-418.el9.x86_64 2024-02-07T12:11:10.311 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:11:10.311 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-02-07T12:11:10.828 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:11:10.963 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T12:11:10.963 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T12:11:10.964 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-418.el9.x86_64 2024-02-07T12:11:10.964 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-07T12:11:10.964 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-07T12:11:10.964 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-07T12:11:10.964 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi078.front.sepia.ceph.com, path=None, version=distro) 2024-02-07T12:11:10.964 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:11:11.070 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T12:11:11.070 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T12:11:11.070 DEBUG:teuthology.orchestra.run.smithi078:> sudo rpm -qi grub2-tools 2024-02-07T12:11:11.115 INFO:teuthology.orchestra.run.smithi078.stdout:Name : grub2-tools 2024-02-07T12:11:11.115 INFO:teuthology.orchestra.run.smithi078.stdout:Epoch : 1 2024-02-07T12:11:11.115 INFO:teuthology.orchestra.run.smithi078.stdout:Version : 2.06 2024-02-07T12:11:11.115 INFO:teuthology.orchestra.run.smithi078.stdout:Release : 68.el9 2024-02-07T12:11:11.115 INFO:teuthology.orchestra.run.smithi078.stdout:Architecture: x86_64 2024-02-07T12:11:11.115 INFO:teuthology.orchestra.run.smithi078.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-07T12:11:11.115 INFO:teuthology.orchestra.run.smithi078.stdout:Group : Unspecified 2024-02-07T12:11:11.115 INFO:teuthology.orchestra.run.smithi078.stdout:Size : 8264254 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:License : GPLv3+ 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:Packager : builder@centos.org 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:Vendor : CentOS 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:Summary : Support tools for GRUB. 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:Description : 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-07T12:11:11.116 INFO:teuthology.orchestra.run.smithi078.stdout:hardware devices. 2024-02-07T12:11:11.117 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:11:11.117 INFO:teuthology.orchestra.run.smithi078.stdout:This subpackage provides tools for support of all platforms. 2024-02-07T12:11:11.117 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-07T12:11:11.118 INFO:teuthology.task.kernel:Updating grub on smithi078 to boot 5.14.0-418.el9.x86_64 2024-02-07T12:11:11.118 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-07T12:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:Generating grub configuration file ... 2024-02-07T12:11:11.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-02-07T12:11:11.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:11:13.017 INFO:teuthology.orchestra.run.smithi078.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-07T12:11:13.053 INFO:teuthology.orchestra.run.smithi078.stderr:done 2024-02-07T12:11:13.055 DEBUG:teuthology.orchestra.run.smithi078:> mktemp 2024-02-07T12:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/tmp.MCtU3d7Soz 2024-02-07T12:11:13.072 DEBUG:teuthology.orchestra.run.smithi078:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MCtU3d7Soz 2024-02-07T12:11:13.144 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0666 /tmp/tmp.MCtU3d7Soz 2024-02-07T12:11:13.293 DEBUG:teuthology.orchestra.remote:smithi078:/tmp/tmp.MCtU3d7Soz is 6KB 2024-02-07T12:11:13.341 DEBUG:teuthology.orchestra.run.smithi078:> rm -fr /tmp/tmp.MCtU3d7Soz 2024-02-07T12:11:13.356 DEBUG:teuthology.orchestra.run.smithi078:> sudo /bin/ls /boot/loader/entries || true 2024-02-07T12:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-07T12:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-07T12:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-07T12:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout:15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64.conf 2024-02-07T12:11:13.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64 2024-02-07T12:11:13.590 DEBUG:teuthology.orchestra.run.smithi078:> sudo shutdown -r now 2024-02-07T12:11:14.908 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-02-07T12:11:24.860 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-02-07T12:11:26.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-02-07T12:11:26.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:11:33.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-02-07T12:11:33.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:11:43.601 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-07T12:11:43.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-07T12:11:43.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:12:01.980 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-02-07T12:12:10.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-07T12:12:10.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:12:14.044 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-02-07T12:12:26.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-07T12:12:26.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:12:26.915 DEBUG:teuthology.orchestra.remote:timed out 2024-02-07T12:12:33.874 DEBUG:teuthology.orchestra.remote:timed out 2024-02-07T12:12:41.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-02-07T12:12:41.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:12:42.393 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-02-07T12:12:42.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-02-07T12:12:42.801 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-418.el9.x86_64"... 2024-02-07T12:12:42.801 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-02-07T12:12:42.832 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-418.el9.x86_64 2024-02-07T12:12:42.833 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-418.el9.x86_64 vs 5.14.0-418.el9.x86_64 2024-02-07T12:12:42.833 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-07T12:12:42.833 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-07T12:12:43.833 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-02-07T12:12:43.834 DEBUG:teuthology.orchestra.run.smithi171:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-07T12:12:43.908 INFO:teuthology.orchestra.run.smithi171.stdout:ttyS1 2024-02-07T12:12:43.938 DEBUG:teuthology.parallel:result is None 2024-02-07T12:12:45.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-02-07T12:12:45.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:12:46.284 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-02-07T12:12:46.681 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-02-07T12:12:46.682 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-418.el9.x86_64"... 2024-02-07T12:12:46.682 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-02-07T12:12:46.698 INFO:teuthology.orchestra.run.smithi106.stdout:5.14.0-418.el9.x86_64 2024-02-07T12:12:46.698 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-418.el9.x86_64 vs 5.14.0-418.el9.x86_64 2024-02-07T12:12:46.698 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-07T12:12:46.698 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-07T12:12:47.699 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-07T12:12:47.699 DEBUG:teuthology.orchestra.run.smithi106:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-07T12:12:47.785 INFO:teuthology.orchestra.run.smithi106.stdout:ttyS1 2024-02-07T12:12:47.826 DEBUG:teuthology.parallel:result is None 2024-02-07T12:12:57.911 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-02-07T12:12:58.392 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-07T12:12:58.393 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-418.el9.x86_64"... 2024-02-07T12:12:58.393 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-02-07T12:12:58.406 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-418.el9.x86_64 2024-02-07T12:12:58.407 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-418.el9.x86_64 vs 5.14.0-418.el9.x86_64 2024-02-07T12:12:58.407 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-07T12:12:58.407 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-07T12:12:59.408 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-07T12:12:59.408 DEBUG:teuthology.orchestra.run.smithi078:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-07T12:12:59.520 INFO:teuthology.orchestra.run.smithi078.stdout:ttyS1 2024-02-07T12:12:59.580 DEBUG:teuthology.parallel:result is None 2024-02-07T12:12:59.580 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-07T12:12:59.587 INFO:teuthology.task.internal:Creating test directory... 2024-02-07T12:12:59.588 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-07T12:12:59.590 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-07T12:12:59.593 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-07T12:12:59.611 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-07T12:12:59.618 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-07T12:12:59.626 INFO:teuthology.task.internal:Creating archive directory... 2024-02-07T12:12:59.626 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-07T12:12:59.658 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-07T12:12:59.660 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-07T12:12:59.701 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-07T12:12:59.708 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-07T12:12:59.708 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-07T12:12:59.744 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-07T12:12:59.746 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-07T12:12:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T12:12:59.788 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T12:12:59.798 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T12:12:59.802 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T12:12:59.871 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T12:12:59.891 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T12:12:59.893 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-07T12:12:59.900 INFO:teuthology.task.internal:Configuring sudo... 2024-02-07T12:12:59.900 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-07T12:12:59.904 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-07T12:12:59.906 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-07T12:12:59.944 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-07T12:12:59.958 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-07T12:12:59.958 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-07T12:12:59.991 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-07T12:12:59.994 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-07T12:13:00.012 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T12:13:00.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T12:13:00.181 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T12:13:00.230 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T12:13:00.294 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:13:00.294 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-07T12:13:00.365 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T12:13:00.411 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T12:13:00.469 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T12:13:00.553 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T12:13:00.598 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:13:00.598 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-07T12:13:00.668 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T12:13:00.715 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T12:13:00.776 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T12:13:00.818 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T12:13:00.864 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:13:00.864 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-07T12:13:00.933 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-02-07T12:13:00.936 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-02-07T12:13:00.939 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-02-07T12:13:00.982 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T12:13:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T12:13:01.013 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T12:13:01.333 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-07T12:13:01.343 INFO:teuthology.task.internal:Starting timer... 2024-02-07T12:13:01.343 INFO:teuthology.run_tasks:Running task pcp... 2024-02-07T12:13:01.354 INFO:teuthology.run_tasks:Running task selinux... 2024-02-07T12:13:01.363 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-07T12:13:01.363 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-02-07T12:13:01.421 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-02-07T12:13:01.422 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-02-07T12:13:01.483 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-02-07T12:13:01.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-02-07T12:13:01.533 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-02-07T12:13:01.535 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-07T12:13:01.535 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-02-07T12:13:01.556 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-02-07T12:13:01.556 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-02-07T12:13:01.582 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-02-07T12:13:01.583 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-02-07T12:13:01.604 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-02-07T12:13:01.605 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-02-07T12:13:01.606 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T12:13:01.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:13:01.633 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T12:13:01.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:13:01.664 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T12:13:01.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:13:01.692 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-07T12:13:01.692 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-02-07T12:13:01.725 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-02-07T12:13:01.760 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-02-07T12:13:01.804 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-07T12:13:01.814 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-07T12:13:01.835 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-07T12:13:01.837 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi106.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-07T12:17:20.185 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-02-07T12:17:20.187 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-07T12:17:20.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:17:20.281 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-02-07T12:17:20.363 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-07T12:17:20.363 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-02-07T12:17:20.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:17:20.451 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-02-07T12:17:20.535 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-02-07T12:17:20.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-02-07T12:17:20.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:17:20.624 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-02-07T12:17:20.707 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-02-07T12:17:20.708 INFO:teuthology.run_tasks:Running task clock... 2024-02-07T12:17:20.720 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-07T12:17:20.720 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-07T12:17:20.720 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T12:17:20.723 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-07T12:17:20.723 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T12:17:20.725 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-07T12:17:20.725 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T12:17:20.751 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-07T12:17:20.755 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-07T12:17:20.767 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-07T12:17:20.772 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-07T12:17:20.780 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-07T12:17:20.796 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-02-07T12:17:20.797 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-07T12:17:20.802 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-02-07T12:17:20.809 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-02-07T12:17:20.815 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-02-07T12:17:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-07T12:17:20.828 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-02-07T12:17:20.832 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-07T12:17:20.841 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-02-07T12:17:20.842 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-07T12:17:20.848 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-07T12:17:20.858 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-07T12:17:20.874 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-07T12:17:20.895 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-02-07T12:17:20.898 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T12:17:20.898 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-02-07T12:17:20.898 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.898 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.898 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.899 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-02-07T12:17:20.903 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T12:17:20.903 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-02-07T12:17:20.903 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.903 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.903 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.903 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.930 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-02-07T12:17:20.934 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T12:17:20.934 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-02-07T12:17:20.934 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.934 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.934 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.934 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:17:20.934 INFO:teuthology.run_tasks:Running task pexec... 2024-02-07T12:17:20.944 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-07T12:17:20.945 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi078.front.sepia.ceph.com 2024-02-07T12:17:20.945 DEBUG:teuthology.orchestra.run.smithi078:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-07T12:17:20.946 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi106.front.sepia.ceph.com 2024-02-07T12:17:20.946 DEBUG:teuthology.orchestra.run.smithi106:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-07T12:17:20.946 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi171.front.sepia.ceph.com 2024-02-07T12:17:20.946 DEBUG:teuthology.orchestra.run.smithi171:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-07T12:17:21.531 INFO:teuthology.orchestra.run.smithi106.stdout:Last metadata expiration check: 0:00:58 ago on Wed 07 Feb 2024 12:16:23 PM UTC. 2024-02-07T12:17:21.536 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:58 ago on Wed 07 Feb 2024 12:16:23 PM UTC. 2024-02-07T12:17:21.562 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:49 ago on Wed 07 Feb 2024 12:16:32 PM UTC. 2024-02-07T12:17:21.684 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-02-07T12:17:21.690 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-02-07T12:17:21.694 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout: Package Architecture Version Repository Size 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 3.1 M 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 9.8 M 2024-02-07T12:17:21.695 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-02-07T12:17:21.703 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-07T12:17:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-02-07T12:17:21.703 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-07T12:17:21.703 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-02-07T12:17:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-07T12:17:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:17:21.704 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-02-07T12:17:21.704 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-07T12:17:21.704 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-02-07T12:17:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:17:21.706 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 3.1 M 2024-02-07T12:17:21.706 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 9.8 M 2024-02-07T12:17:21.706 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-02-07T12:17:21.716 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-02-07T12:17:21.718 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-02-07T12:17:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-02-07T12:17:21.718 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-02-07T12:17:21.718 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-02-07T12:17:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-07T12:17:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:17:21.719 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-02-07T12:17:21.719 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-02-07T12:17:21.719 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-02-07T12:17:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:17:21.719 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 3.1 M 2024-02-07T12:17:21.719 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 9.8 M 2024-02-07T12:17:21.719 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-02-07T12:17:21.934 INFO:teuthology.orchestra.run.smithi106.stdout:runc-1.1.12-1.el9.x86_64.rpm 13 MB/s | 3.1 MB 00:00 2024-02-07T12:17:21.934 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:17:21.935 INFO:teuthology.orchestra.run.smithi106.stdout:Total 13 MB/s | 3.1 MB 00:00 2024-02-07T12:17:21.935 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-02-07T12:17:21.940 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-02-07T12:17:21.940 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-02-07T12:17:21.944 INFO:teuthology.orchestra.run.smithi078.stdout:runc-1.1.12-1.el9.x86_64.rpm 13 MB/s | 3.1 MB 00:00 2024-02-07T12:17:21.944 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:17:21.945 INFO:teuthology.orchestra.run.smithi078.stdout:Total 13 MB/s | 3.1 MB 00:00 2024-02-07T12:17:21.945 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-02-07T12:17:21.950 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-02-07T12:17:21.950 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-02-07T12:17:21.968 INFO:teuthology.orchestra.run.smithi171.stdout:runc-1.1.12-1.el9.x86_64.rpm 13 MB/s | 3.1 MB 00:00 2024-02-07T12:17:21.969 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:17:21.969 INFO:teuthology.orchestra.run.smithi171.stdout:Total 12 MB/s | 3.1 MB 00:00 2024-02-07T12:17:21.969 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-02-07T12:17:21.974 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-02-07T12:17:21.975 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-02-07T12:17:22.010 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-02-07T12:17:22.011 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-02-07T12:17:22.021 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-02-07T12:17:22.021 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-02-07T12:17:22.046 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-02-07T12:17:22.046 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-02-07T12:17:22.218 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-02-07T12:17:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-02-07T12:17:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-02-07T12:17:22.312 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T12:17:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T12:17:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T12:17:22.839 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T12:17:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T12:17:22.916 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T12:17:23.160 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T12:17:23.160 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:17:23.160 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-02-07T12:17:23.160 INFO:teuthology.orchestra.run.smithi078.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-07T12:17:23.160 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:17:23.160 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-02-07T12:17:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T12:17:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:17:23.219 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-02-07T12:17:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-07T12:17:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:17:23.219 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-02-07T12:17:23.223 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T12:17:23.223 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:17:23.223 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-02-07T12:17:23.223 INFO:teuthology.orchestra.run.smithi106.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-07T12:17:23.224 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-02-07T12:17:23.224 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-02-07T12:17:23.432 DEBUG:teuthology.parallel:result is None 2024-02-07T12:17:23.450 DEBUG:teuthology.parallel:result is None 2024-02-07T12:17:23.462 DEBUG:teuthology.parallel:result is None 2024-02-07T12:17:23.462 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-02-07T12:17:23.473 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-02-07T12:17:23.473 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:17:23.473 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T12:17:23.488 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-07T12:17:23.488 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-02-07T12:17:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-07T12:17:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:23.544 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-02-07T12:17:23.544 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:23.544 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:23.544 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-07 12:17:19.173173573 +0000 2024-02-07T12:17:23.544 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-07 12:16:47.999676774 +0000 2024-02-07T12:17:23.544 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-07 12:16:47.999676774 +0000 2024-02-07T12:17:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-07 12:16:47.999676774 +0000 2024-02-07T12:17:23.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-07T12:17:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-07T12:17:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-07T12:17:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000261081 s, 2.0 MB/s 2024-02-07T12:17:23.611 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-07T12:17:23.667 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-02-07T12:17:23.722 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-07T12:17:23.722 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:23.722 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-07T12:17:23.722 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:23.722 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:23.722 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-07 12:17:19.173173573 +0000 2024-02-07T12:17:23.722 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-07 12:16:48.349671124 +0000 2024-02-07T12:17:23.722 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-07 12:16:48.349671124 +0000 2024-02-07T12:17:23.722 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-07 12:16:48.349671124 +0000 2024-02-07T12:17:23.723 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-07T12:17:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-07T12:17:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-07T12:17:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000201137 s, 2.5 MB/s 2024-02-07T12:17:23.788 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-07T12:17:23.843 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-02-07T12:17:23.898 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-07T12:17:23.899 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:23.899 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-07T12:17:23.899 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:23.899 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:23.899 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-07 12:17:19.173173573 +0000 2024-02-07T12:17:23.899 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-07 12:16:48.699665474 +0000 2024-02-07T12:17:23.899 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-07 12:16:48.699665474 +0000 2024-02-07T12:17:23.899 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-07 12:16:48.699665474 +0000 2024-02-07T12:17:23.899 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-07T12:17:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-07T12:17:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-07T12:17:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000256103 s, 2.0 MB/s 2024-02-07T12:17:23.966 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-07T12:17:24.021 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-02-07T12:17:24.076 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-07T12:17:24.076 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:24.076 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-07T12:17:24.076 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:24.076 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:24.076 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-07 12:17:19.173173573 +0000 2024-02-07T12:17:24.076 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-07 12:16:49.039659986 +0000 2024-02-07T12:17:24.076 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-07 12:16:49.039659986 +0000 2024-02-07T12:17:24.077 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-07 12:16:49.039659986 +0000 2024-02-07T12:17:24.077 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-07T12:17:24.141 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-07T12:17:24.141 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-07T12:17:24.141 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000188958 s, 2.7 MB/s 2024-02-07T12:17:24.143 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-07T12:17:24.199 DEBUG:teuthology.orchestra.run.smithi078:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-07T12:17:24.333 INFO:teuthology.orchestra.run.smithi078.stdout:loop 2024-02-07T12:17:24.335 INFO:tasks.nvme_loop:Connecting nvme_loop smithi078:/dev/vg_nvme/lv_1... 2024-02-07T12:17:24.335 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-07T12:17:24.381 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-02-07T12:17:24.425 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/vg_nvme/lv_11 2024-02-07T12:17:24.458 INFO:tasks.nvme_loop:Connecting nvme_loop smithi078:/dev/vg_nvme/lv_2... 2024-02-07T12:17:24.458 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-07T12:17:24.537 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-02-07T12:17:24.579 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/vg_nvme/lv_21 2024-02-07T12:17:24.620 INFO:tasks.nvme_loop:Connecting nvme_loop smithi078:/dev/vg_nvme/lv_3... 2024-02-07T12:17:24.620 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-07T12:17:24.699 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-02-07T12:17:24.738 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/vg_nvme/lv_31 2024-02-07T12:17:24.766 INFO:tasks.nvme_loop:Connecting nvme_loop smithi078:/dev/vg_nvme/lv_4... 2024-02-07T12:17:24.766 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-07T12:17:24.845 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-02-07T12:17:24.884 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/vg_nvme/lv_41 2024-02-07T12:17:24.914 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:17:24.914 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T12:17:24.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme list 2024-02-07T12:17:25.037 INFO:teuthology.orchestra.run.smithi078.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-07T12:17:25.037 INFO:teuthology.orchestra.run.smithi078.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-07T12:17:25.037 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme0n1 PHFT620400QR400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-07T12:17:25.037 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme1n1 87964fc57034fcb8e951 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:25.037 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme2n1 9f094c66bb0f6918158b Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:25.037 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme3n1 db7cd0359163b82d7693 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:25.038 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme4n1 a99c2fd380d19918a907 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:25.038 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T12:17:25.038 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:17:25.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/scratch_devs 2024-02-07T12:17:25.106 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:17:25.106 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T12:17:25.125 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-07T12:17:25.125 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-02-07T12:17:25.182 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-07T12:17:25.182 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:25.183 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-07T12:17:25.183 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:25.183 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:25.183 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-02-07 12:17:19.196289657 +0000 2024-02-07T12:17:25.183 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-02-07 12:16:47.994041700 +0000 2024-02-07T12:17:25.183 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-02-07 12:16:47.994041700 +0000 2024-02-07T12:17:25.183 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-02-07 12:16:47.994041700 +0000 2024-02-07T12:17:25.183 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-07T12:17:25.253 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-02-07T12:17:25.253 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-02-07T12:17:25.253 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000249777 s, 2.0 MB/s 2024-02-07T12:17:25.255 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-07T12:17:25.313 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-02-07T12:17:25.372 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-07T12:17:25.372 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:25.372 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-07T12:17:25.372 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:25.372 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:25.373 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-02-07 12:17:19.196289657 +0000 2024-02-07T12:17:25.373 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-02-07 12:16:48.345033241 +0000 2024-02-07T12:17:25.373 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-02-07 12:16:48.345033241 +0000 2024-02-07T12:17:25.373 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-02-07 12:16:48.345033241 +0000 2024-02-07T12:17:25.373 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-07T12:17:25.440 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-02-07T12:17:25.440 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-02-07T12:17:25.441 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000219855 s, 2.3 MB/s 2024-02-07T12:17:25.441 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-07T12:17:25.500 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-02-07T12:17:25.556 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-07T12:17:25.556 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:25.556 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-07T12:17:25.556 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:25.556 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:25.556 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-02-07 12:17:19.196289657 +0000 2024-02-07T12:17:25.556 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-02-07 12:16:48.683025095 +0000 2024-02-07T12:17:25.556 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-02-07 12:16:48.683025095 +0000 2024-02-07T12:17:25.557 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-02-07 12:16:48.683025095 +0000 2024-02-07T12:17:25.557 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-07T12:17:25.624 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-02-07T12:17:25.624 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-02-07T12:17:25.624 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000192043 s, 2.7 MB/s 2024-02-07T12:17:25.625 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-07T12:17:25.682 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-02-07T12:17:25.737 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-07T12:17:25.738 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:25.738 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-07T12:17:25.738 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:25.738 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:25.738 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-02-07 12:17:19.196289657 +0000 2024-02-07T12:17:25.738 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-02-07 12:16:49.025016852 +0000 2024-02-07T12:17:25.738 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-02-07 12:16:49.025016852 +0000 2024-02-07T12:17:25.738 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-02-07 12:16:49.025016852 +0000 2024-02-07T12:17:25.738 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-07T12:17:25.804 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-02-07T12:17:25.804 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-02-07T12:17:25.804 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000214085 s, 2.4 MB/s 2024-02-07T12:17:25.805 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-07T12:17:25.862 DEBUG:teuthology.orchestra.run.smithi106:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-07T12:17:25.996 INFO:teuthology.orchestra.run.smithi106.stdout:loop 2024-02-07T12:17:25.997 INFO:tasks.nvme_loop:Connecting nvme_loop smithi106:/dev/vg_nvme/lv_1... 2024-02-07T12:17:25.997 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-07T12:17:26.034 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-02-07T12:17:26.072 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/vg_nvme/lv_11 2024-02-07T12:17:26.112 INFO:tasks.nvme_loop:Connecting nvme_loop smithi106:/dev/vg_nvme/lv_2... 2024-02-07T12:17:26.112 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-07T12:17:26.204 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-02-07T12:17:26.243 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/vg_nvme/lv_21 2024-02-07T12:17:26.280 INFO:tasks.nvme_loop:Connecting nvme_loop smithi106:/dev/vg_nvme/lv_3... 2024-02-07T12:17:26.280 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-07T12:17:26.319 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-02-07T12:17:26.358 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/vg_nvme/lv_31 2024-02-07T12:17:26.394 INFO:tasks.nvme_loop:Connecting nvme_loop smithi106:/dev/vg_nvme/lv_4... 2024-02-07T12:17:26.394 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-07T12:17:26.473 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-02-07T12:17:26.512 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/vg_nvme/lv_41 2024-02-07T12:17:26.542 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:17:26.542 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T12:17:26.598 DEBUG:teuthology.orchestra.run.smithi106:> sudo nvme list 2024-02-07T12:17:26.665 INFO:teuthology.orchestra.run.smithi106.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-07T12:17:26.665 INFO:teuthology.orchestra.run.smithi106.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-07T12:17:26.665 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/nvme0n1 PHFT6204013R400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-07T12:17:26.665 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/nvme1n1 dc0d812a6363dc70dd80 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:26.665 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/nvme2n1 a746df0f88e09fa5eb0f Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:26.666 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/nvme3n1 353d0ef6347aaf7ebc45 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:26.666 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/nvme4n1 186f5d0517d4df19c475 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:26.666 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T12:17:26.667 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:17:26.667 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/scratch_devs 2024-02-07T12:17:26.734 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:17:26.734 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T12:17:26.753 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-07T12:17:26.753 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-02-07T12:17:26.810 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-07T12:17:26.811 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:26.811 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-07T12:17:26.811 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:26.811 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:26.811 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-02-07 12:17:19.167827753 +0000 2024-02-07T12:17:26.811 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-02-07 12:16:56.847255726 +0000 2024-02-07T12:17:26.811 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-02-07 12:16:56.847255726 +0000 2024-02-07T12:17:26.811 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-02-07 12:16:56.847255726 +0000 2024-02-07T12:17:26.811 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-07T12:17:26.879 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-02-07T12:17:26.880 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-02-07T12:17:26.880 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000335052 s, 1.5 MB/s 2024-02-07T12:17:26.881 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-07T12:17:26.938 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-02-07T12:17:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-07T12:17:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:26.995 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-07T12:17:26.995 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:26.995 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:26.995 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-02-07 12:17:19.167827753 +0000 2024-02-07T12:17:26.995 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-02-07 12:16:57.210248766 +0000 2024-02-07T12:17:26.995 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-02-07 12:16:57.210248766 +0000 2024-02-07T12:17:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-02-07 12:16:57.210248766 +0000 2024-02-07T12:17:26.995 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-07T12:17:27.062 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-02-07T12:17:27.062 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-02-07T12:17:27.062 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000220214 s, 2.3 MB/s 2024-02-07T12:17:27.064 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-07T12:17:27.120 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-02-07T12:17:27.176 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-07T12:17:27.176 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:27.176 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-07T12:17:27.176 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:27.176 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:27.176 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-02-07 12:17:19.167827753 +0000 2024-02-07T12:17:27.176 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-02-07 12:16:57.586241556 +0000 2024-02-07T12:17:27.176 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-02-07 12:16:57.586241556 +0000 2024-02-07T12:17:27.176 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-02-07 12:16:57.586241556 +0000 2024-02-07T12:17:27.176 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-07T12:17:27.242 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-02-07T12:17:27.242 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-02-07T12:17:27.242 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000230929 s, 2.2 MB/s 2024-02-07T12:17:27.243 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-07T12:17:27.300 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-02-07T12:17:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-07T12:17:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T12:17:27.355 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-07T12:17:27.355 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T12:17:27.356 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T12:17:27.356 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-02-07 12:17:19.167827753 +0000 2024-02-07T12:17:27.356 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-02-07 12:16:57.934234884 +0000 2024-02-07T12:17:27.356 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-02-07 12:16:57.934234884 +0000 2024-02-07T12:17:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-02-07 12:16:57.934234884 +0000 2024-02-07T12:17:27.356 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-07T12:17:27.421 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-02-07T12:17:27.421 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-02-07T12:17:27.421 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000185025 s, 2.8 MB/s 2024-02-07T12:17:27.422 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-07T12:17:27.478 DEBUG:teuthology.orchestra.run.smithi171:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-07T12:17:27.621 INFO:teuthology.orchestra.run.smithi171.stdout:loop 2024-02-07T12:17:27.622 INFO:tasks.nvme_loop:Connecting nvme_loop smithi171:/dev/vg_nvme/lv_1... 2024-02-07T12:17:27.623 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-07T12:17:27.660 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-02-07T12:17:27.697 INFO:teuthology.orchestra.run.smithi171.stdout:/dev/vg_nvme/lv_11 2024-02-07T12:17:27.733 INFO:tasks.nvme_loop:Connecting nvme_loop smithi171:/dev/vg_nvme/lv_2... 2024-02-07T12:17:27.734 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-07T12:17:27.818 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-02-07T12:17:27.857 INFO:teuthology.orchestra.run.smithi171.stdout:/dev/vg_nvme/lv_21 2024-02-07T12:17:27.898 INFO:tasks.nvme_loop:Connecting nvme_loop smithi171:/dev/vg_nvme/lv_3... 2024-02-07T12:17:27.898 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-07T12:17:27.936 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-02-07T12:17:27.974 INFO:teuthology.orchestra.run.smithi171.stdout:/dev/vg_nvme/lv_31 2024-02-07T12:17:28.003 INFO:tasks.nvme_loop:Connecting nvme_loop smithi171:/dev/vg_nvme/lv_4... 2024-02-07T12:17:28.004 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-07T12:17:28.082 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-02-07T12:17:28.123 INFO:teuthology.orchestra.run.smithi171.stdout:/dev/vg_nvme/lv_41 2024-02-07T12:17:28.151 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:17:28.151 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T12:17:28.205 DEBUG:teuthology.orchestra.run.smithi171:> sudo nvme list 2024-02-07T12:17:28.271 INFO:teuthology.orchestra.run.smithi171.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-07T12:17:28.271 INFO:teuthology.orchestra.run.smithi171.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-07T12:17:28.271 INFO:teuthology.orchestra.run.smithi171.stdout:/dev/nvme0n1 CVFT6233001S400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-07T12:17:28.272 INFO:teuthology.orchestra.run.smithi171.stdout:/dev/nvme1n1 ad5d6d85b611333c766c Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:28.272 INFO:teuthology.orchestra.run.smithi171.stdout:/dev/nvme2n1 3736b4043a6b92f3e579 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:28.272 INFO:teuthology.orchestra.run.smithi171.stdout:/dev/nvme3n1 bd6d59cceaa621335d02 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:28.272 INFO:teuthology.orchestra.run.smithi171.stdout:/dev/nvme4n1 a8bc866201e75415c106 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T12:17:28.273 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T12:17:28.273 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:17:28.273 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/scratch_devs 2024-02-07T12:17:28.341 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-07T12:17:28.441 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '44458ccedb18241cc2e17f10cdab3d1385399261'} 2024-02-07T12:17:28.441 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T12:17:28.442 INFO:tasks.cephadm:Cluster fsid is dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:17:28.442 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-07T12:17:28.442 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.78', 'mon.b': '172.21.15.106', 'mon.c': '172.21.15.171'} 2024-02-07T12:17:28.442 INFO:tasks.cephadm:First mon is mon.a on smithi078 2024-02-07T12:17:28.442 INFO:tasks.cephadm:First mgr is y 2024-02-07T12:17:28.443 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-07T12:17:28.443 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-02-07T12:17:28.476 DEBUG:teuthology.orchestra.run.smithi106:> sudo hostname $(hostname -s) 2024-02-07T12:17:28.508 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-02-07T12:17:28.535 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-07T12:17:28.536 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T12:17:28.679 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '44458ccedb18241cc2e17f10cdab3d1385399261', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76537/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1252-g44458cce', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1252.g44458cce'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-05 21:02:35.601617', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-testing-4-2024-02-05-1854', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-07T12:17:28.896 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-guits-testing-4-2024-02-05-1854, sha1 44458ccedb18241cc2e17f10cdab3d1385399261 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T12:17:28.898 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm 2024-02-07T12:17:28.898 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-07T12:17:29.456 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 766878 Feb 7 12:17 /home/ubuntu/cephtest/cephadm 2024-02-07T12:17:29.807 DEBUG:teuthology.orchestra.run.smithi106:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-07T12:17:30.222 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r--. 1 ubuntu ubuntu 766878 Feb 7 12:17 /home/ubuntu/cephtest/cephadm 2024-02-07T12:17:30.222 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-07T12:17:30.633 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 766878 Feb 7 12:17 /home/ubuntu/cephtest/cephadm 2024-02-07T12:17:30.633 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-07T12:17:30.652 DEBUG:teuthology.orchestra.run.smithi106:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-07T12:17:30.671 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-07T12:17:30.714 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 on all hosts... 2024-02-07T12:17:30.715 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 pull 2024-02-07T12:17:30.717 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 pull 2024-02-07T12:17:30.720 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 pull 2024-02-07T12:17:31.486 INFO:teuthology.orchestra.run.smithi106.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T12:17:31.562 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T12:17:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T12:18:08.329 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-02-07T12:18:08.329 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev)", 2024-02-07T12:18:08.329 INFO:teuthology.orchestra.run.smithi106.stdout: "image_id": "79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f", 2024-02-07T12:18:08.330 INFO:teuthology.orchestra.run.smithi106.stdout: "repo_digests": [ 2024-02-07T12:18:08.330 INFO:teuthology.orchestra.run.smithi106.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe" 2024-02-07T12:18:08.330 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-02-07T12:18:08.330 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-02-07T12:18:13.676 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-02-07T12:18:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev)", 2024-02-07T12:18:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f", 2024-02-07T12:18:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-02-07T12:18:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe" 2024-02-07T12:18:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-02-07T12:18:13.677 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-02-07T12:18:18.043 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-02-07T12:18:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev)", 2024-02-07T12:18:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f", 2024-02-07T12:18:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-02-07T12:18:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe" 2024-02-07T12:18:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-02-07T12:18:18.044 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-02-07T12:18:18.069 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-02-07T12:18:18.104 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph 2024-02-07T12:18:18.136 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-02-07T12:18:18.166 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-02-07T12:18:18.196 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /etc/ceph 2024-02-07T12:18:18.224 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-02-07T12:18:18.252 INFO:tasks.cephadm:Writing seed config... 2024-02-07T12:18:18.253 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-07T12:18:18.253 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-07T12:18:18.254 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-02-07T12:18:18.254 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-07T12:18:18.254 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-07T12:18:18.254 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-07T12:18:18.254 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-07T12:18:18.254 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-07T12:18:18.254 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-07T12:18:18.254 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-07T12:18:18.255 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:18:18.255 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-07T12:18:18.272 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = dc35b7c8-c5b2-11ee-95b6-87774f69a715 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-02-07T12:18:18.273 DEBUG:teuthology.orchestra.run.smithi078:mon.a> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.a.service 2024-02-07T12:18:18.315 DEBUG:teuthology.orchestra.run.smithi078:mgr.y> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.y.service 2024-02-07T12:18:18.358 INFO:tasks.cephadm:Bootstrapping... 2024-02-07T12:18:18.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 -v bootstrap --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.78 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-07T12:18:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:18:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261', '-v', 'bootstrap', '--fsid', 'dc35b7c8-c5b2-11ee-95b6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.78', '--skip-admin-label'] 2024-02-07T12:18:18.574 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-07T12:18:18.575 INFO:teuthology.orchestra.run.smithi078.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-07T12:18:18.575 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying podman|docker is present... 2024-02-07T12:18:18.598 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-07T12:18:18.598 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying lvm2 is present... 2024-02-07T12:18:18.599 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying time synchronization is in place... 2024-02-07T12:18:18.605 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-07T12:18:18.605 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-07T12:18:18.611 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-07T12:18:18.611 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-02-07T12:18:18.617 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-02-07T12:18:18.623 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-02-07T12:18:18.623 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-02-07T12:18:18.623 INFO:teuthology.orchestra.run.smithi078.stdout:Repeating the final host check... 2024-02-07T12:18:18.646 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-07T12:18:18.646 INFO:teuthology.orchestra.run.smithi078.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-07T12:18:18.646 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl is present 2024-02-07T12:18:18.647 INFO:teuthology.orchestra.run.smithi078.stdout:lvcreate is present 2024-02-07T12:18:18.652 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-07T12:18:18.652 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-07T12:18:18.658 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-07T12:18:18.658 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-02-07T12:18:18.664 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-02-07T12:18:18.671 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-02-07T12:18:18.671 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-02-07T12:18:18.671 INFO:teuthology.orchestra.run.smithi078.stdout:Host looks OK 2024-02-07T12:18:18.671 INFO:teuthology.orchestra.run.smithi078.stdout:Cluster fsid: dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:18:18.671 INFO:teuthology.orchestra.run.smithi078.stdout:Acquiring lock 140229267823200 on /run/cephadm/dc35b7c8-c5b2-11ee-95b6-87774f69a715.lock 2024-02-07T12:18:18.671 INFO:teuthology.orchestra.run.smithi078.stdout:Lock 140229267823200 acquired on /run/cephadm/dc35b7c8-c5b2-11ee-95b6-87774f69a715.lock 2024-02-07T12:18:18.671 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 3300 ... 2024-02-07T12:18:18.672 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 6789 ... 2024-02-07T12:18:18.672 INFO:teuthology.orchestra.run.smithi078.stdout:Base mon IP(s) is [172.21.15.78:3300, 172.21.15.78:6789], mon addrv is [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-02-07T12:18:18.674 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.78 metric 100 2024-02-07T12:18:18.674 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.78 metric 100 2024-02-07T12:18:18.681 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-07T12:18:18.681 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-07T12:18:18.681 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-02-07T12:18:18.681 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-07T12:18:18.681 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-07T12:18:18.681 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-07T12:18:18.681 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-07T12:18:18.681 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-02-07T12:18:18.681 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-07T12:18:18.682 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-02-07T12:18:18.682 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-02-07T12:18:18.682 INFO:teuthology.orchestra.run.smithi078.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-07T12:18:18.682 INFO:teuthology.orchestra.run.smithi078.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-07T12:18:18.682 INFO:teuthology.orchestra.run.smithi078.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T12:18:19.169 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f 2024-02-07T12:18:19.169 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T12:18:19.170 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Getting image source signatures 2024-02-07T12:18:19.170 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:a997f0e9fc346761d24d2a6e2e6c8569e51b56d4493dc8639d7f291d92935b5f 2024-02-07T12:18:19.170 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-07T12:18:19.170 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying config sha256:79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f 2024-02-07T12:18:19.170 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-07T12:18:20.125 INFO:teuthology.orchestra.run.smithi078.stdout:ceph: stdout ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev) 2024-02-07T12:18:20.125 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph version: ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev) 2024-02-07T12:18:20.125 INFO:teuthology.orchestra.run.smithi078.stdout:Extracting ceph user uid/gid from container image... 2024-02-07T12:18:21.229 INFO:teuthology.orchestra.run.smithi078.stdout:stat: stdout 167 167 2024-02-07T12:18:21.229 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial keys... 2024-02-07T12:18:22.218 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQANdcNlWCAOIxAACRMFmQ0Y+fbQaNyWT7Z09Q== 2024-02-07T12:18:23.207 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQAOdcNl3rXqIRAAp6G++s0/RTdLOI1+po/TWg== 2024-02-07T12:18:24.103 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQAPdcNlfqK5IBAArMkvdiogHJch2D2/8bWKNw== 2024-02-07T12:18:24.103 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial monmap... 2024-02-07T12:18:25.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-07T12:18:25.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-07T12:18:25.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:18:25.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-07T12:18:25.092 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool for a [v2:172.21.15.78:3300,v1:172.21.15.78:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-07T12:18:25.093 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = quincy 2024-02-07T12:18:25.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: set fsid to dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:18:25.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-07T12:18:25.093 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:18:25.093 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mon... 2024-02-07T12:18:25.957 INFO:teuthology.orchestra.run.smithi078.stdout:create mon.a on 2024-02-07T12:18:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-07T12:18:26.519 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715.target. 2024-02-07T12:18:26.519 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715.target. 2024-02-07T12:18:26.750 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.a 2024-02-07T12:18:26.750 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.a.service: Unit ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.a.service not loaded. 2024-02-07T12:18:26.911 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715.target.wants/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@.service. 2024-02-07T12:18:27.263 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:27 smithi078 podman[25049]: 2024-02-07 12:18:27.050487399 +0000 UTC m=+0.017261618 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T12:18:27.437 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T12:18:27.437 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-02-07T12:18:27.444 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T12:18:27.444 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-02-07T12:18:27.444 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-02-07T12:18:27.445 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-02-07T12:18:27.445 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon to start... 2024-02-07T12:18:27.445 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon... 2024-02-07T12:18:27.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:27 smithi078 podman[25049]: 2024-02-07 12:18:27.355801504 +0000 UTC m=+0.322575722 container init ca8aebfd6e9e09dcd0546cb4fda542bcfddc45a136df74c2cab52760f6a314cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131) 2024-02-07T12:18:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:27 smithi078 podman[25049]: 2024-02-07 12:18:27.361565608 +0000 UTC m=+0.328339826 container start ca8aebfd6e9e09dcd0546cb4fda542bcfddc45a136df74c2cab52760f6a314cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, ceph=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T12:18:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:27 smithi078 bash[25049]: ca8aebfd6e9e09dcd0546cb4fda542bcfddc45a136df74c2cab52760f6a314cd 2024-02-07T12:18:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:27 smithi078 systemd[1]: Started Ceph mon.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:18:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:27 smithi078 ceph-mon[25087]: mkfs dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:18:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:27 smithi078 ceph-mon[25087]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-07T12:18:28.559 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout cluster: 2024-02-07T12:18:28.559 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout id: dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:18:28.559 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-07T12:18:28.559 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:28.559 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout services: 2024-02-07T12:18:28.559 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.576467s) 2024-02-07T12:18:28.559 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout data: 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pgs: 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:mon is available 2024-02-07T12:18:28.560 INFO:teuthology.orchestra.run.smithi078.stdout:Assimilating anything we can from ceph.conf... 2024-02-07T12:18:28.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:28 smithi078 ceph-mon[25087]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-07T12:18:28.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:28 smithi078 ceph-mon[25087]: monmap e1: 1 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:18:28.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:28 smithi078 ceph-mon[25087]: fsmap 2024-02-07T12:18:28.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:28 smithi078 ceph-mon[25087]: osdmap e1: 0 total, 0 up, 0 in 2024-02-07T12:18:28.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:28 smithi078 ceph-mon[25087]: mgrmap e1: no daemons active 2024-02-07T12:18:28.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:28 smithi078 ceph-mon[25087]: from='client.? 172.21.15.78:0/1472452894' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-07T12:18:29.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:29 smithi078 ceph-mon[25087]: from='client.? 172.21.15.78:0/3802761671' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-07T12:18:29.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:29 smithi078 ceph-mon[25087]: from='client.? 172.21.15.78:0/3802761671' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:29.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-07T12:18:29.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-02-07T12:18:29.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-07T12:18:29.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:29.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-02-07T12:18:29.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-07T12:18:29.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-07T12:18:29.775 INFO:teuthology.orchestra.run.smithi078.stdout:Generating new minimal ceph.conf... 2024-02-07T12:18:30.809 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:30 smithi078 ceph-mon[25087]: from='client.? 172.21.15.78:0/164486728' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:18:30.811 INFO:teuthology.orchestra.run.smithi078.stdout:Restarting the monitor... 2024-02-07T12:18:31.081 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:30 smithi078 systemd[1]: Stopping Ceph mon.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:18:31.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:31 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a[25064]: 2024-02-07T12:18:31.078+0000 7f783c323700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T12:18:31.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:31 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a[25064]: 2024-02-07T12:18:31.078+0000 7f783c323700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-07T12:18:31.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:31 smithi078 podman[25419]: 2024-02-07 12:18:31.243213352 +0000 UTC m=+0.270582859 container died ca8aebfd6e9e09dcd0546cb4fda542bcfddc45a136df74c2cab52760f6a314cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-07T12:18:31.736 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:31 smithi078 podman[25419]: 2024-02-07 12:18:31.500903972 +0000 UTC m=+0.528273478 container cleanup ca8aebfd6e9e09dcd0546cb4fda542bcfddc45a136df74c2cab52760f6a314cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T12:18:31.737 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:31 smithi078 bash[25419]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a 2024-02-07T12:18:32.062 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:31 smithi078 podman[25438]: 2024-02-07 12:18:31.734995845 +0000 UTC m=+0.490233403 container remove ca8aebfd6e9e09dcd0546cb4fda542bcfddc45a136df74c2cab52760f6a314cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-07T12:18:32.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:31 smithi078 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-07T12:18:32.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:31 smithi078 systemd[1]: Stopped Ceph mon.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:18:32.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:31 smithi078 systemd[1]: Starting Ceph mon.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:18:32.313 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 podman[25538]: 2024-02-07 12:18:32.060932122 +0000 UTC m=+0.106302805 container create d62527df0e17bc417459490415bc0199e777110253aa043c14409e2175b457e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-07T12:18:32.313 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 podman[25538]: 2024-02-07 12:18:31.971624677 +0000 UTC m=+0.016995368 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T12:18:32.314 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 podman[25538]: 2024-02-07 12:18:32.279886828 +0000 UTC m=+0.325257510 container init d62527df0e17bc417459490415bc0199e777110253aa043c14409e2175b457e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, ceph=True, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T12:18:32.314 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 podman[25538]: 2024-02-07 12:18:32.285167725 +0000 UTC m=+0.330538410 container start d62527df0e17bc417459490415bc0199e777110253aa043c14409e2175b457e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-02-07T12:18:32.314 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: set uid:gid to 167:167 (ceph:ceph) 2024-02-07T12:18:32.314 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev), process ceph-mon, pid 7 2024-02-07T12:18:32.314 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: pidfile_write: ignore empty --pid-file 2024-02-07T12:18:32.314 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: load: jerasure load: lrc 2024-02-07T12:18:32.402 INFO:teuthology.orchestra.run.smithi078.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-07T12:18:32.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: RocksDB version: 7.9.2 2024-02-07T12:18:32.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Git sha 0 2024-02-07T12:18:32.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Compile date 2024-02-05 19:41:51 2024-02-07T12:18:32.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: DB SUMMARY 2024-02-07T12:18:32.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: DB Session ID: OOHWGEB20VNMY3QZIRMK 2024-02-07T12:18:32.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: CURRENT file: CURRENT 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: IDENTITY file: IDENTITY 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 84623 ; 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.error_if_exists: 0 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.create_if_missing: 0 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.paranoid_checks: 1 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.env: 0x5581141afae0 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.fs: PosixFileSystem 2024-02-07T12:18:32.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.info_log: 0x558114bb98c0 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_file_opening_threads: 16 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.statistics: (nil) 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.use_fsync: 0 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_log_file_size: 0 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.keep_log_file_num: 1000 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.recycle_log_file_num: 0 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.allow_fallocate: 1 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.allow_mmap_reads: 0 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.allow_mmap_writes: 0 2024-02-07T12:18:32.634 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.use_direct_reads: 0 2024-02-07T12:18:32.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-07T12:18:32.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.create_missing_column_families: 0 2024-02-07T12:18:32.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.db_log_dir: 2024-02-07T12:18:32.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.wal_dir: 2024-02-07T12:18:32.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-07T12:18:32.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-07T12:18:32.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-07T12:18:32.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.advise_random_on_open: 1 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.db_write_buffer_size: 0 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.write_buffer_manager: 0x558114b8d400 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.rate_limiter: (nil) 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.wal_recovery_mode: 2 2024-02-07T12:18:32.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.enable_thread_tracking: 0 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.enable_pipelined_write: 0 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.unordered_write: 0 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.row_cache: None 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.wal_filter: None 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.allow_ingest_behind: 0 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.two_write_queues: 0 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.manual_wal_flush: 0 2024-02-07T12:18:32.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.wal_compression: 0 2024-02-07T12:18:32.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.atomic_flush: 0 2024-02-07T12:18:32.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-07T12:18:32.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-07T12:18:32.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-07T12:18:32.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.log_readahead_size: 0 2024-02-07T12:18:32.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.best_efforts_recovery: 0 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.allow_data_in_errors: 0 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.db_host_id: __hostname__ 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_background_jobs: 2 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_background_compactions: -1 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_subcompactions: 1 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-07T12:18:32.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_total_wal_size: 0 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_open_files: -1 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bytes_per_sync: 0 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_readahead_size: 0 2024-02-07T12:18:32.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_background_flushes: -1 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Compression algorithms supported: 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: kZSTD supported: 0 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: kXpressCompression supported: 0 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: kLZ4HCCompression supported: 1 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: kZlibCompression supported: 1 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: kSnappyCompression supported: 1 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: kLZ4Compression supported: 1 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: kBZip2Compression supported: 0 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-07T12:18:32.642 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.merge_operator: 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_filter: None 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_filter_factory: None 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.sst_partitioner_factory: None 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558114bb99c0) 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_type: 0 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_index_type: 0 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_shortening: 1 2024-02-07T12:18:32.643 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: checksum: 4 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: no_block_cache: 0 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache: 0x558114bbd090 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_options: 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: capacity : 536870912 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_shard_bits : 4 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: strict_capacity_limit : 0 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_compressed: (nil) 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: persistent_cache: (nil) 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size: 4096 2024-02-07T12:18:32.644 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size_deviation: 10 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_restart_interval: 16 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_block_restart_interval: 1 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: metadata_block_size: 4096 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: partition_filters: 0 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: use_delta_encoding: 1 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: filter_policy: bloomfilter 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: whole_key_filtering: 1 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: verify_compression: 0 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: format_version: 5 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_index_compression: 1 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_align: 0 2024-02-07T12:18:32.645 INFO:journalctl@ceph.mon.a.smithi078.stdout: max_auto_readahead_size: 262144 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout: prepopulate_block_cache: 0 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout: initial_auto_readahead_size: 8192 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.write_buffer_size: 33554432 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_write_buffer_number: 2 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression: NoCompression 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression: Disabled 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.prefix_extractor: nullptr 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.num_levels: 7 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-07T12:18:32.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression_opts.level: 32767 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression_opts.strategy: 0 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-07T12:18:32.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression_opts.enabled: false 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.target_file_size_base: 67108864 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-07T12:18:32.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.arena_block_size: 1048576 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.disable_auto_compactions: 0 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-07T12:18:32.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.table_properties_collectors: 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.inplace_update_support: 0 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-07T12:18:32.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.bloom_locality: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.max_successive_merges: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.paranoid_file_checks: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.force_consistency_checks: 1 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.report_bg_io_stats: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.ttl: 2592000 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.enable_blob_files: false 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.min_blob_size: 0 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.blob_file_size: 268435456 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-07T12:18:32.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.blob_file_starting_level: 0 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dd6f21dc-e823-4c61-bfb3-40152c38779a 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707308312313579, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707308312314415, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81248, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 252, "table_properties": {"data_size": 79435, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 73744, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 211, "num_filter_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707308312, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dd6f21dc-e823-4c61-bfb3-40152c38779a", "db_session_id": "OOHWGEB20VNMY3QZIRMK", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707308312314494, "job": 1, "event": "recovery_finished"} 2024-02-07T12:18:32.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558114cae000 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: DB pointer 0x558114c98000 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** DB Stats ** 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T12:18:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: L0 2/0 81.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: Sum 2/0 81.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 148.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-02-07T12:18:32.654 INFO:journalctl@ceph.mon.a.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative compaction: 0.00 GB write, 27.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval compaction: 0.00 GB write, 27.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache BinnedLRUCache@0x558114bbd090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-07T12:18:32.655 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: starting mon.a rank 0 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mon.a@-1(???) e1 preinit fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mon.a@-1(???).mds e1 new map 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mon.a@-1(???).mds e1 print_map 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout: e1 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout: legacy client fscid: -1 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout: No filesystems configured 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-07T12:18:32.656 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: monmap e1: 1 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: fsmap 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: osdmap e1: 0 total, 0 up, 0 in 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 ceph-mon[25576]: mgrmap e1: no daemons active 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 bash[25538]: d62527df0e17bc417459490415bc0199e777110253aa043c14409e2175b457e9 2024-02-07T12:18:32.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:32 smithi078 systemd[1]: Started Ceph mon.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:18:33.523 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-07T12:18:33.545 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-07T12:18:33.545 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mgr... 2024-02-07T12:18:33.546 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-07T12:18:33.546 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-07T12:18:33.800 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.y 2024-02-07T12:18:33.800 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.y.service: Unit ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.y.service not loaded. 2024-02-07T12:18:34.011 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715.target.wants/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@.service. 2024-02-07T12:18:34.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:33 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3983432063' entity='client.admin' 2024-02-07T12:18:34.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:34 smithi078 systemd[1]: Starting Ceph mgr.y for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:18:34.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:34 smithi078 podman[25815]: 2024-02-07 12:18:34.142785205 +0000 UTC m=+0.017626654 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T12:18:35.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:35 smithi078 podman[25815]: 2024-02-07 12:18:35.263893234 +0000 UTC m=+1.138734667 container create fa19eb12e0b53e5d624f4103e63f0d8632575c88b7b7d4f462bcf93d80d6c83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2) 2024-02-07T12:18:37.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:37 smithi078 podman[25815]: 2024-02-07 12:18:37.130451563 +0000 UTC m=+3.005293009 container init fa19eb12e0b53e5d624f4103e63f0d8632575c88b7b7d4f462bcf93d80d6c83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y, org.label-schema.build-date=20240131, ceph=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T12:18:37.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:37 smithi078 podman[25815]: 2024-02-07 12:18:37.136824353 +0000 UTC m=+3.011665793 container start fa19eb12e0b53e5d624f4103e63f0d8632575c88b7b7d4f462bcf93d80d6c83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True) 2024-02-07T12:18:38.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:37 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:37.680+0000 7faeb45ac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T12:18:38.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:37 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:37.771+0000 7faeb45ac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T12:18:38.196 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T12:18:38.196 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-02-07T12:18:38.203 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T12:18:38.203 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-02-07T12:18:38.203 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-02-07T12:18:38.203 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-02-07T12:18:38.212 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T12:18:38.212 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-02-07T12:18:38.219 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T12:18:38.219 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-02-07T12:18:38.220 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-02-07T12:18:38.220 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-07T12:18:38.220 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr to start... 2024-02-07T12:18:38.220 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr... 2024-02-07T12:18:38.485 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:38 smithi078 bash[25815]: fa19eb12e0b53e5d624f4103e63f0d8632575c88b7b7d4f462bcf93d80d6c83c 2024-02-07T12:18:38.485 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:38 smithi078 systemd[1]: Started Ceph mgr.y for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:18:38.485 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:38 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:38.421+0000 7faeb45ac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T12:18:38.748 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:38 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:38.512+0000 7faeb45ac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T12:18:38.748 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:38 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:38.651+0000 7faeb45ac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T12:18:38.748 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:38 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:38.745+0000 7faeb45ac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T12:18:39.087 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:38 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:38.827+0000 7faeb45ac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T12:18:39.087 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:39 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:39.084+0000 7faeb45ac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T12:18:39.372 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:39 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:39.193+0000 7faeb45ac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T12:18:39.372 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:39 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:39.277+0000 7faeb45ac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T12:18:39.372 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:39 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:39.370+0000 7faeb45ac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T12:18:39.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:39 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:39.543+0000 7faeb45ac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T12:18:40.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:40 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:40.520+0000 7faeb45ac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T12:18:40.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:40 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:40.660+0000 7faeb45ac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T12:18:41.243 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:40 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:40.988+0000 7faeb45ac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T12:18:41.244 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:41 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:41.079+0000 7faeb45ac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T12:18:41.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:41 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4269329818' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T12:18:41.554 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:41 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:41.283+0000 7faeb45ac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:18:41.554 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:41 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:41.467+0000 7faeb45ac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T12:18:41.854 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:41 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:41.752+0000 7faeb45ac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:18:41.863 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "dc35b7c8-c5b2-11ee-95b6-87774f69a715", 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T12:18:41.864 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T12:18:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T12:18:41.866 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T12:18:41.866 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T12:18:41.866 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T12:18:41.866 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:41.866 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T12:18:41.866 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T12:18:41.866 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-02-07T12:18:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T12:18:41.869 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:41.869 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T12:18:41.869 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:41.869 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T12:18:27.440228+0000", 2024-02-07T12:18:41.869 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T12:18:41.869 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:41.869 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T12:18:41.869 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-02-07T12:18:41.869 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (1/15)... 2024-02-07T12:18:42.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:42 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:42.011+0000 7faeb45ac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T12:18:42.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:42 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:42.242+0000 7faeb45ac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T12:18:43.380 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:43 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:43.030+0000 7faeb45ac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T12:18:43.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:43 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:43.619+0000 7faeb45ac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T12:18:43.980 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:43 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:43.721+0000 7faeb45ac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T12:18:43.980 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:43 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:43.944+0000 7faeb45ac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T12:18:44.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: Activating manager daemon y 2024-02-07T12:18:44.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: mgrmap e2: y(active, starting, since 0.00311185s) 2024-02-07T12:18:44.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T12:18:44.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T12:18:44.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T12:18:44.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:18:44.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T12:18:44.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: Manager daemon y is now available 2024-02-07T12:18:44.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T12:18:44.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T12:18:44.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' 2024-02-07T12:18:44.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' 2024-02-07T12:18:44.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:44 smithi078 ceph-mon[25576]: from='mgr.14100 172.21.15.78:0/386699065' entity='mgr.y' 2024-02-07T12:18:45.012 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:45.012 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-02-07T12:18:45.012 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "dc35b7c8-c5b2-11ee-95b6-87774f69a715", 2024-02-07T12:18:45.012 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T12:18:45.012 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T12:18:45.012 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T12:18:45.013 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T12:18:45.014 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T12:18:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T12:18:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T12:18:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T12:18:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T12:18:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T12:18:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T12:18:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T12:18:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T12:18:45.016 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T12:18:45.016 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T12:18:45.016 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:45.016 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T12:18:45.016 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T12:18:45.017 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:45.018 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T12:18:45.018 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:45.018 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T12:18:27.440228+0000", 2024-02-07T12:18:45.018 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T12:18:45.018 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:45.018 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T12:18:45.018 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-02-07T12:18:45.018 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (2/15)... 2024-02-07T12:18:45.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:45 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4209797310' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T12:18:45.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:45 smithi078 ceph-mon[25576]: mgrmap e3: y(active, since 1.00581s) 2024-02-07T12:18:47.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:47 smithi078 ceph-mon[25576]: mgrmap e4: y(active, since 2s) 2024-02-07T12:18:49.626 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:49 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3076065198' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T12:18:49.910 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:49.910 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "dc35b7c8-c5b2-11ee-95b6-87774f69a715", 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T12:18:49.911 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T12:18:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:49.914 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T12:18:49.915 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T12:18:49.915 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T12:18:27.440228+0000", 2024-02-07T12:18:49.915 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T12:18:49.915 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-02-07T12:18:49.915 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T12:18:49.915 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-02-07T12:18:49.915 INFO:teuthology.orchestra.run.smithi078.stdout:mgr is available 2024-02-07T12:18:50.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:50 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3714732713' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-07T12:18:50.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:50 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3714732713' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-07T12:18:51.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-07T12:18:51.095 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling cephadm module... 2024-02-07T12:18:52.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:51 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2479070372' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-07T12:18:52.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:51 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ignoring --setuser ceph since I am not root 2024-02-07T12:18:52.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:51 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ignoring --setgroup ceph since I am not root 2024-02-07T12:18:52.573 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:52 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:52.380+0000 7f466dab1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T12:18:52.573 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:52 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:52.476+0000 7f466dab1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T12:18:53.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:52 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2479070372' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-07T12:18:53.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:52 smithi078 ceph-mon[25576]: mgrmap e5: y(active, since 7s) 2024-02-07T12:18:53.467 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:53 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:53.129+0000 7f466dab1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T12:18:53.467 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:53 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:53.226+0000 7f466dab1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T12:18:53.467 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:53 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:53.367+0000 7f466dab1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T12:18:53.467 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:53 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:53.464+0000 7f466dab1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T12:18:53.781 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:53 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:53.548+0000 7f466dab1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T12:18:54.105 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:53 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/572432128' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-07T12:18:54.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:53 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:53.815+0000 7f466dab1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T12:18:54.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:53 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:53.924+0000 7f466dab1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T12:18:54.106 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:54 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:54.009+0000 7f466dab1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T12:18:54.106 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:54 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:54.102+0000 7f466dab1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T12:18:54.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-02-07T12:18:54.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-07T12:18:54.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-07T12:18:54.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-07T12:18:54.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-07T12:18:54.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-02-07T12:18:54.159 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-02-07T12:18:54.159 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 5... 2024-02-07T12:18:54.375 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:54 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:54.284+0000 7f466dab1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T12:18:55.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:55 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:55.255+0000 7f466dab1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T12:18:55.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:55 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:55.384+0000 7f466dab1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T12:18:55.998 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:55 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:55.713+0000 7f466dab1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T12:18:55.998 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:55 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:55.802+0000 7f466dab1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T12:18:55.998 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:55 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:55.995+0000 7f466dab1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:18:56.381 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:56 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:56.178+0000 7f466dab1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T12:18:56.721 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:56 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:56.461+0000 7f466dab1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:18:56.721 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:56 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:56.718+0000 7f466dab1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T12:18:57.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:56 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:56.952+0000 7f466dab1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T12:18:58.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:57.742+0000 7f466dab1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T12:18:58.619 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:58.306+0000 7f466dab1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T12:18:58.619 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:58.403+0000 7f466dab1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T12:18:58.620 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:18:58.616+0000 7f466dab1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T12:18:58.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: Active manager daemon y restarted 2024-02-07T12:18:58.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: Activating manager daemon y 2024-02-07T12:18:58.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: osdmap e2: 0 total, 0 up, 0 in 2024-02-07T12:18:58.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: mgrmap e6: y(active, starting, since 0.00330275s) 2024-02-07T12:18:58.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:18:58.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T12:18:58.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T12:18:58.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T12:18:58.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T12:18:58.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: Manager daemon y is now available 2024-02-07T12:18:58.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:18:58.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:18:58.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:18:58.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:58 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:18:59.986 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:59 smithi078 ceph-mon[25576]: Found migration_current of "None". Setting to last migration. 2024-02-07T12:18:59.986 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:59 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:18:59.986 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:59 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T12:18:59.986 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:59 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T12:18:59.986 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:59 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:18:59.986 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:59 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:18:59.986 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:18:59 smithi078 ceph-mon[25576]: mgrmap e7: y(active, since 1.00518s) 2024-02-07T12:19:00.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-02-07T12:19:00.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-07T12:19:00.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-07T12:19:00.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-02-07T12:19:00.239 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 5 is available 2024-02-07T12:19:00.239 INFO:teuthology.orchestra.run.smithi078.stdout:Setting orchestrator backend to cephadm... 2024-02-07T12:19:00.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:00 smithi078 ceph-mon[25576]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-07T12:19:00.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:00 smithi078 ceph-mon[25576]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-07T12:19:00.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:00 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:01.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:01 smithi078 ceph-mon[25576]: [07/Feb/2024:12:19:00] ENGINE Bus STARTING 2024-02-07T12:19:01.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:01 smithi078 ceph-mon[25576]: [07/Feb/2024:12:19:00] ENGINE Serving on http://172.21.15.78:8765 2024-02-07T12:19:01.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:01 smithi078 ceph-mon[25576]: [07/Feb/2024:12:19:00] ENGINE Serving on https://172.21.15.78:7150 2024-02-07T12:19:01.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:01 smithi078 ceph-mon[25576]: [07/Feb/2024:12:19:00] ENGINE Bus STARTED 2024-02-07T12:19:01.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:01 smithi078 ceph-mon[25576]: mgrmap e8: y(active, since 2s) 2024-02-07T12:19:01.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:01 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:19:01.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:01 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:03.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:02 smithi078 ceph-mon[25576]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:03.018 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-07T12:19:03.018 INFO:teuthology.orchestra.run.smithi078.stdout:Generating ssh key... 2024-02-07T12:19:04.104 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-mon[25576]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: Generating public/private rsa key pair. 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: Your identification has been saved in /tmp/tmpbm2tnvf3/key. 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: Your public key has been saved in /tmp/tmpbm2tnvf3/key.pub. 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: The key fingerprint is: 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: SHA256:G561GSKARmeoutTQrzr4ynsqy2Bi/FtXh813na8+6HI ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: The key's randomart image is: 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: +---[RSA 3072]----+ 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: | ..o | 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: | ..+ | 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: | .+ . | 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: |.o . . + o| 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: |. o . . Soo+ . o.| 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: |o. . . o.*.+. . .| 2024-02-07T12:19:04.105 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: |== .. .+ o . .| 2024-02-07T12:19:04.106 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: |@..o. . . E .. | 2024-02-07T12:19:04.106 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: |+XBo. +..o. | 2024-02-07T12:19:04.106 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:03 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: +----[SHA256]-----+ 2024-02-07T12:19:05.082 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:04 smithi078 ceph-mon[25576]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:05.082 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:04 smithi078 ceph-mon[25576]: Generating ssh key... 2024-02-07T12:19:05.082 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:04 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:19:05.082 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:04 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:19:05.612 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChLUWvlPdwZgqPfHGVdm4nwzZjsOlVbyZHBUTE7vF3c3/7ZFI5v8HelW0WdvW2x6Zc+1wa7PqgWx94ccKryEos7WccCu+zfvbDuPXx8mHUbrzWDvubxfYOXYpUZVnBE9t3b5jTI4ysJnXJnXbjZaiIB20A+eStDDhPHxmD13CH/IjifYgzf7BulKzm1Pp0N5lV+F7IlQ0TuzlkFDfYvJ3r80qNy2l4a6Ko6uLXtLkH40xeDQEGjA2HMGQRtZtizn/Pr3iowYiA2b/7ypffmdfhboRRjdgUzhHiS6DhC/YFegY+NWQ8pYk7aanXiIZdS55yU9onbGu/+Ea+Dw1+gAZKHtLO6k1WWnepnq05VOo+ouPY6xnjElVmyZrEOgCRdVrr/VeDGOIMlTbJECyRFm8cE9ilL0xi8zB+DOAxpBBeSw5S4NoPvDyED2/2AWSlJTSDc6DjzIdnQg7AIkUBbTDUinYFpYVEUOWlXYTif404q4iGREZGBzGVLlu4aYjVQss= ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:19:05.613 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-07T12:19:05.613 INFO:teuthology.orchestra.run.smithi078.stdout:Adding key to root@localhost authorized_keys... 2024-02-07T12:19:05.613 INFO:teuthology.orchestra.run.smithi078.stdout:Adding host smithi078... 2024-02-07T12:19:05.988 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:05 smithi078 ceph-mon[25576]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:07.189 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:06 smithi078 ceph-mon[25576]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:08.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:08 smithi078 ceph-mon[25576]: Deploying cephadm binary to smithi078 2024-02-07T12:19:11.170 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Added host 'smithi078' with addr '172.21.15.78' 2024-02-07T12:19:11.170 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mon service... 2024-02-07T12:19:11.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:11 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:19:11.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:11 smithi078 ceph-mon[25576]: Added host smithi078 2024-02-07T12:19:11.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:11 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:12.714 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-07T12:19:12.714 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mgr service... 2024-02-07T12:19:12.961 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:12 smithi078 ceph-mon[25576]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:12.961 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:12 smithi078 ceph-mon[25576]: Saving service mon spec with placement count:5 2024-02-07T12:19:12.961 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:12 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:19:14.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-07T12:19:14.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:14 smithi078 ceph-mon[25576]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:14.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:14 smithi078 ceph-mon[25576]: Saving service mgr spec with placement count:2 2024-02-07T12:19:14.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:14 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:19:16.056 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:15 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3554318394' entity='client.admin' 2024-02-07T12:19:16.056 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:15 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:19:16.056 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:15 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:19:17.063 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling the dashboard module... 2024-02-07T12:19:17.313 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:17 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1169938605' entity='client.admin' 2024-02-07T12:19:19.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:18 smithi078 ceph-mon[25576]: from='mgr.14118 172.21.15.78:0/1380396545' entity='mgr.y' 2024-02-07T12:19:19.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:18 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3781712152' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-07T12:19:19.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:18 smithi078 ceph-mon[25576]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:19:19.236 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ignoring --setuser ceph since I am not root 2024-02-07T12:19:19.236 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ignoring --setgroup ceph since I am not root 2024-02-07T12:19:19.535 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:19 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:19.462+0000 7fb817661200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T12:19:19.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:19 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:19.561+0000 7fb817661200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T12:19:20.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:19 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3781712152' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-07T12:19:20.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:19 smithi078 ceph-mon[25576]: mgrmap e9: y(active, since 20s) 2024-02-07T12:19:20.561 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:20 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:20.218+0000 7fb817661200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T12:19:20.561 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:20 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:20.319+0000 7fb817661200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T12:19:20.561 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:20 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:20.460+0000 7fb817661200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T12:19:20.561 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:20 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:20.558+0000 7fb817661200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T12:19:20.841 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:20 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:20.643+0000 7fb817661200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T12:19:20.845 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-02-07T12:19:20.845 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-07T12:19:20.845 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-07T12:19:20.845 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-07T12:19:20.845 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-07T12:19:20.845 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-02-07T12:19:20.845 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-02-07T12:19:20.845 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 9... 2024-02-07T12:19:21.109 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:20 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1308997682' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-07T12:19:21.110 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:20 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:20.910+0000 7fb817661200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T12:19:21.110 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:21 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:21.020+0000 7fb817661200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T12:19:21.110 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:21 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:21.107+0000 7fb817661200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T12:19:21.381 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:21 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:21.205+0000 7fb817661200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T12:19:21.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:21 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:21.386+0000 7fb817661200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T12:19:22.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:22 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:22.342+0000 7fb817661200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T12:19:22.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:22 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:22.473+0000 7fb817661200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T12:19:23.094 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:22 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:22.805+0000 7fb817661200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T12:19:23.094 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:22 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:22.895+0000 7fb817661200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T12:19:23.094 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:23.091+0000 7fb817661200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:19:23.381 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:23.276+0000 7fb817661200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T12:19:23.825 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:23.565+0000 7fb817661200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:19:23.826 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:23.823+0000 7fb817661200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T12:19:24.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:24 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:24.060+0000 7fb817661200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T12:19:25.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:24 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:24.859+0000 7fb817661200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T12:19:25.756 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:25.431+0000 7fb817661200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T12:19:25.756 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:25.531+0000 7fb817661200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T12:19:25.756 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:19:25.753+0000 7fb817661200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T12:19:26.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: Active manager daemon y restarted 2024-02-07T12:19:26.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: Activating manager daemon y 2024-02-07T12:19:26.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: osdmap e3: 0 total, 0 up, 0 in 2024-02-07T12:19:26.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: mgrmap e10: y(active, starting, since 0.00321062s) 2024-02-07T12:19:26.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:19:26.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T12:19:26.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T12:19:26.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T12:19:26.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T12:19:26.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: Manager daemon y is now available 2024-02-07T12:19:26.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:26.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:27.122 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T12:19:27.122 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T12:19:27.122 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:26 smithi078 ceph-mon[25576]: mgrmap e11: y(active, since 1.00593s) 2024-02-07T12:19:27.620 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-02-07T12:19:27.620 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-07T12:19:27.620 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-07T12:19:27.620 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-02-07T12:19:27.620 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 9 is available 2024-02-07T12:19:27.620 INFO:teuthology.orchestra.run.smithi078.stdout:Generating a dashboard self-signed certificate... 2024-02-07T12:19:28.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:27 smithi078 ceph-mon[25576]: [07/Feb/2024:12:19:26] ENGINE Bus STARTING 2024-02-07T12:19:28.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:27 smithi078 ceph-mon[25576]: [07/Feb/2024:12:19:26] ENGINE Serving on http://172.21.15.78:8765 2024-02-07T12:19:28.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:27 smithi078 ceph-mon[25576]: [07/Feb/2024:12:19:26] ENGINE Serving on https://172.21.15.78:7150 2024-02-07T12:19:28.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:27 smithi078 ceph-mon[25576]: [07/Feb/2024:12:19:26] ENGINE Bus STARTED 2024-02-07T12:19:28.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:27 smithi078 ceph-mon[25576]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-07T12:19:28.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:27 smithi078 ceph-mon[25576]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-07T12:19:28.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:29.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-07T12:19:29.134 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial admin user... 2024-02-07T12:19:29.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:29 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:29.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:29 smithi078 ceph-mon[25576]: mgrmap e12: y(active, since 2s) 2024-02-07T12:19:29.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:29 smithi078 ceph-mon[25576]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:29.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:29 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:29.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:29 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:30.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:30 smithi078 ceph-mon[25576]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:30.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:30 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:31.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$v4A/GBR2v9lRQXcPAwLbOu0CNBfJDwP99IYyU/OcgNS4Yzvkpd/rG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707308370, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-07T12:19:31.296 INFO:teuthology.orchestra.run.smithi078.stdout:Fetching dashboard port number... 2024-02-07T12:19:32.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 8443 2024-02-07T12:19:32.520 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T12:19:32.520 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-02-07T12:19:32.527 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T12:19:32.527 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-02-07T12:19:32.527 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-02-07T12:19:32.527 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-07T12:19:32.529 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph Dashboard is now available at: 2024-02-07T12:19:32.529 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:32.529 INFO:teuthology.orchestra.run.smithi078.stdout: URL: https://smithi078.front.sepia.ceph.com:8443/ 2024-02-07T12:19:32.529 INFO:teuthology.orchestra.run.smithi078.stdout: User: admin 2024-02-07T12:19:32.529 INFO:teuthology.orchestra.run.smithi078.stdout: Password: b8ksyji13n 2024-02-07T12:19:32.529 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:32.530 INFO:teuthology.orchestra.run.smithi078.stdout:Saving cluster configuration to /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config directory 2024-02-07T12:19:32.530 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling autotune for osd_memory_target 2024-02-07T12:19:33.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:32 smithi078 ceph-mon[25576]: mgrmap e13: y(active, since 6s) 2024-02-07T12:19:33.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:32 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/149584893' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout:Or, if you are only running a single cluster on this host: 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-07T12:19:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: ceph telemetry on 2024-02-07T12:19:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:35.148 INFO:teuthology.orchestra.run.smithi078.stdout:For more information see: 2024-02-07T12:19:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-07T12:19:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:35.148 INFO:teuthology.orchestra.run.smithi078.stdout:Bootstrap complete. 2024-02-07T12:19:35.188 INFO:tasks.cephadm:Fetching config... 2024-02-07T12:19:35.189 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:19:35.189 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-07T12:19:35.204 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-07T12:19:35.204 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:19:35.205 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-07T12:19:35.259 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-07T12:19:35.259 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:19:35.260 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-07T12:19:35.326 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-07T12:19:35.326 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:19:35.327 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-07T12:19:35.381 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-07T12:19:35.381 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChLUWvlPdwZgqPfHGVdm4nwzZjsOlVbyZHBUTE7vF3c3/7ZFI5v8HelW0WdvW2x6Zc+1wa7PqgWx94ccKryEos7WccCu+zfvbDuPXx8mHUbrzWDvubxfYOXYpUZVnBE9t3b5jTI4ysJnXJnXbjZaiIB20A+eStDDhPHxmD13CH/IjifYgzf7BulKzm1Pp0N5lV+F7IlQ0TuzlkFDfYvJ3r80qNy2l4a6Ko6uLXtLkH40xeDQEGjA2HMGQRtZtizn/Pr3iowYiA2b/7ypffmdfhboRRjdgUzhHiS6DhC/YFegY+NWQ8pYk7aanXiIZdS55yU9onbGu/+Ea+Dw1+gAZKHtLO6k1WWnepnq05VOo+ouPY6xnjElVmyZrEOgCRdVrr/VeDGOIMlTbJECyRFm8cE9ilL0xi8zB+DOAxpBBeSw5S4NoPvDyED2/2AWSlJTSDc6DjzIdnQg7AIkUBbTDUinYFpYVEUOWlXYTif404q4iGREZGBzGVLlu4aYjVQss= ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-07T12:19:35.461 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChLUWvlPdwZgqPfHGVdm4nwzZjsOlVbyZHBUTE7vF3c3/7ZFI5v8HelW0WdvW2x6Zc+1wa7PqgWx94ccKryEos7WccCu+zfvbDuPXx8mHUbrzWDvubxfYOXYpUZVnBE9t3b5jTI4ysJnXJnXbjZaiIB20A+eStDDhPHxmD13CH/IjifYgzf7BulKzm1Pp0N5lV+F7IlQ0TuzlkFDfYvJ3r80qNy2l4a6Ko6uLXtLkH40xeDQEGjA2HMGQRtZtizn/Pr3iowYiA2b/7ypffmdfhboRRjdgUzhHiS6DhC/YFegY+NWQ8pYk7aanXiIZdS55yU9onbGu/+Ea+Dw1+gAZKHtLO6k1WWnepnq05VOo+ouPY6xnjElVmyZrEOgCRdVrr/VeDGOIMlTbJECyRFm8cE9ilL0xi8zB+DOAxpBBeSw5S4NoPvDyED2/2AWSlJTSDc6DjzIdnQg7AIkUBbTDUinYFpYVEUOWlXYTif404q4iGREZGBzGVLlu4aYjVQss= ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:19:35.474 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChLUWvlPdwZgqPfHGVdm4nwzZjsOlVbyZHBUTE7vF3c3/7ZFI5v8HelW0WdvW2x6Zc+1wa7PqgWx94ccKryEos7WccCu+zfvbDuPXx8mHUbrzWDvubxfYOXYpUZVnBE9t3b5jTI4ysJnXJnXbjZaiIB20A+eStDDhPHxmD13CH/IjifYgzf7BulKzm1Pp0N5lV+F7IlQ0TuzlkFDfYvJ3r80qNy2l4a6Ko6uLXtLkH40xeDQEGjA2HMGQRtZtizn/Pr3iowYiA2b/7ypffmdfhboRRjdgUzhHiS6DhC/YFegY+NWQ8pYk7aanXiIZdS55yU9onbGu/+Ea+Dw1+gAZKHtLO6k1WWnepnq05VOo+ouPY6xnjElVmyZrEOgCRdVrr/VeDGOIMlTbJECyRFm8cE9ilL0xi8zB+DOAxpBBeSw5S4NoPvDyED2/2AWSlJTSDc6DjzIdnQg7AIkUBbTDUinYFpYVEUOWlXYTif404q4iGREZGBzGVLlu4aYjVQss= ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-07T12:19:35.521 INFO:teuthology.orchestra.run.smithi106.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChLUWvlPdwZgqPfHGVdm4nwzZjsOlVbyZHBUTE7vF3c3/7ZFI5v8HelW0WdvW2x6Zc+1wa7PqgWx94ccKryEos7WccCu+zfvbDuPXx8mHUbrzWDvubxfYOXYpUZVnBE9t3b5jTI4ysJnXJnXbjZaiIB20A+eStDDhPHxmD13CH/IjifYgzf7BulKzm1Pp0N5lV+F7IlQ0TuzlkFDfYvJ3r80qNy2l4a6Ko6uLXtLkH40xeDQEGjA2HMGQRtZtizn/Pr3iowYiA2b/7ypffmdfhboRRjdgUzhHiS6DhC/YFegY+NWQ8pYk7aanXiIZdS55yU9onbGu/+Ea+Dw1+gAZKHtLO6k1WWnepnq05VOo+ouPY6xnjElVmyZrEOgCRdVrr/VeDGOIMlTbJECyRFm8cE9ilL0xi8zB+DOAxpBBeSw5S4NoPvDyED2/2AWSlJTSDc6DjzIdnQg7AIkUBbTDUinYFpYVEUOWlXYTif404q4iGREZGBzGVLlu4aYjVQss= ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:19:35.536 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChLUWvlPdwZgqPfHGVdm4nwzZjsOlVbyZHBUTE7vF3c3/7ZFI5v8HelW0WdvW2x6Zc+1wa7PqgWx94ccKryEos7WccCu+zfvbDuPXx8mHUbrzWDvubxfYOXYpUZVnBE9t3b5jTI4ysJnXJnXbjZaiIB20A+eStDDhPHxmD13CH/IjifYgzf7BulKzm1Pp0N5lV+F7IlQ0TuzlkFDfYvJ3r80qNy2l4a6Ko6uLXtLkH40xeDQEGjA2HMGQRtZtizn/Pr3iowYiA2b/7ypffmdfhboRRjdgUzhHiS6DhC/YFegY+NWQ8pYk7aanXiIZdS55yU9onbGu/+Ea+Dw1+gAZKHtLO6k1WWnepnq05VOo+ouPY6xnjElVmyZrEOgCRdVrr/VeDGOIMlTbJECyRFm8cE9ilL0xi8zB+DOAxpBBeSw5S4NoPvDyED2/2AWSlJTSDc6DjzIdnQg7AIkUBbTDUinYFpYVEUOWlXYTif404q4iGREZGBzGVLlu4aYjVQss= ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-07T12:19:35.582 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChLUWvlPdwZgqPfHGVdm4nwzZjsOlVbyZHBUTE7vF3c3/7ZFI5v8HelW0WdvW2x6Zc+1wa7PqgWx94ccKryEos7WccCu+zfvbDuPXx8mHUbrzWDvubxfYOXYpUZVnBE9t3b5jTI4ysJnXJnXbjZaiIB20A+eStDDhPHxmD13CH/IjifYgzf7BulKzm1Pp0N5lV+F7IlQ0TuzlkFDfYvJ3r80qNy2l4a6Ko6uLXtLkH40xeDQEGjA2HMGQRtZtizn/Pr3iowYiA2b/7ypffmdfhboRRjdgUzhHiS6DhC/YFegY+NWQ8pYk7aanXiIZdS55yU9onbGu/+Ea+Dw1+gAZKHtLO6k1WWnepnq05VOo+ouPY6xnjElVmyZrEOgCRdVrr/VeDGOIMlTbJECyRFm8cE9ilL0xi8zB+DOAxpBBeSw5S4NoPvDyED2/2AWSlJTSDc6DjzIdnQg7AIkUBbTDUinYFpYVEUOWlXYTif404q4iGREZGBzGVLlu4aYjVQss= ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:19:35.597 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-07T12:19:35.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:35 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1067622940' entity='client.admin' 2024-02-07T12:19:35.837 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:19:38.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:38 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:38.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:38 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:38.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:38 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:19:38.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:38 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:38.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:38 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:38.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:38 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:38.989 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-07T12:19:38.990 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-07T12:19:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:19:39.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:39 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/280928314' entity='client.admin' 2024-02-07T12:19:39.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:39 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:39.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:39 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:41.849 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi106 2024-02-07T12:19:41.849 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:19:41.849 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.conf 2024-02-07T12:19:41.867 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:19:41.868 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:19:41.925 INFO:tasks.cephadm:Adding host smithi106 to orchestrator... 2024-02-07T12:19:41.925 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch host add smithi106 2024-02-07T12:19:42.164 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:19:42.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:42.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:42.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:42.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:19:42.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:19:42.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-07T12:19:42.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:19:42.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:19:42.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:19:42.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:42.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:42.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:42 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:44.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:44 smithi078 ceph-mon[25576]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:45.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:45 smithi078 ceph-mon[25576]: Deploying cephadm binary to smithi106 2024-02-07T12:19:46.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:46 smithi078 ceph-mon[25576]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:19:48.257 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi106' with addr '172.21.15.106' 2024-02-07T12:19:48.624 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:48 smithi078 ceph-mon[25576]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:19:48.624 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:48.624 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:48.955 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-07T12:19:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:19:49.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:49 smithi078 ceph-mon[25576]: Added host smithi106 2024-02-07T12:19:49.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:49 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:49.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:49 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:51.009 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:19:51.010 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}] 2024-02-07T12:19:51.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:50 smithi078 ceph-mon[25576]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:19:51.579 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-02-07T12:19:51.580 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:19:51.580 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-02-07T12:19:51.598 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:19:51.598 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:19:51.656 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-02-07T12:19:51.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch host add smithi171 2024-02-07T12:19:51.880 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:51 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:19:52.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:52 smithi078 ceph-mon[25576]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:19:52.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:52 smithi078 ceph-mon[25576]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:19:55.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:54 smithi078 ceph-mon[25576]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:19:55.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:54 smithi078 ceph-mon[25576]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:19:55.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:54 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:55.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:54 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:55.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:54 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:55.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:54 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:55.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:54 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:19:55.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:54 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:19:55.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:54 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:19:56.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:55 smithi078 ceph-mon[25576]: Updating smithi106:/etc/ceph/ceph.conf 2024-02-07T12:19:56.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:55 smithi078 ceph-mon[25576]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:19:56.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:55 smithi078 ceph-mon[25576]: Deploying cephadm binary to smithi171 2024-02-07T12:19:56.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:55 smithi078 ceph-mon[25576]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:19:56.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:55 smithi078 ceph-mon[25576]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:19:57.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:56 smithi078 ceph-mon[25576]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:19:57.951 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-02-07T12:19:58.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:58.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:58.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:58.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:58 smithi078 ceph-mon[25576]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:19:58.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:58.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:58 smithi078 ceph-mon[25576]: Added host smithi171 2024-02-07T12:19:58.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:19:58.581 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-07T12:19:58.813 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:19:59.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:59 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:19:59.620 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:19:59 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:00.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:00 smithi078 ceph-mon[25576]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:00.758 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:20:00.758 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-02-07T12:20:01.329 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-07T12:20:01.330 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd crush tunables default 2024-02-07T12:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:20:01.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:01.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:01 smithi078 ceph-mon[25576]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:20:02.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:02 smithi078 ceph-mon[25576]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2024-02-07T12:20:03.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:03 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/13457377' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-07T12:20:04.210 INFO:tasks.cephadm:Adding mon.a on smithi078 2024-02-07T12:20:04.211 INFO:tasks.cephadm:Adding mon.b on smithi106 2024-02-07T12:20:04.211 INFO:tasks.cephadm:Adding mon.c on smithi171 2024-02-07T12:20:04.211 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch apply mon '3;smithi078:172.21.15.78=a;smithi106:172.21.15.106=b;smithi171:172.21.15.171=c' 2024-02-07T12:20:04.445 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/13457377' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-07T12:20:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:04.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:04.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:20:04.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:04.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:04.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: Updating smithi171:/etc/ceph/ceph.conf 2024-02-07T12:20:04.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:04.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:04 smithi078 ceph-mon[25576]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:05.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:05 smithi078 ceph-mon[25576]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:20:05.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:05 smithi078 ceph-mon[25576]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:20:05.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:05 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:05.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:05 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:05.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:05 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:06.100 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mon update... 2024-02-07T12:20:06.670 DEBUG:teuthology.orchestra.run.smithi106:mon.b> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.b.service 2024-02-07T12:20:06.673 DEBUG:teuthology.orchestra.run.smithi171:mon.c> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.c.service 2024-02-07T12:20:06.675 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-07T12:20:06.675 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-07T12:20:06.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:06 smithi078 ceph-mon[25576]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:06.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:06.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:06.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:06.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:06.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:06.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:06.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:07.430 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:07.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:07 smithi078 ceph-mon[25576]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi106:172.21.15.106=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:07.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:07 smithi078 ceph-mon[25576]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi106:172.21.15.106=b;smithi171:172.21.15.171=c;count:3 2024-02-07T12:20:07.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:07 smithi078 ceph-mon[25576]: Deploying daemon mon.c on smithi171 2024-02-07T12:20:08.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:08 smithi078 ceph-mon[25576]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:10.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 systemd[1]: Starting Ceph mon.c for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:20:10.642 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:20:10.642 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":1,"fsid":"dc35b7c8-c5b2-11ee-95b6-87774f69a715","modified":"2024-02-07T12:18:24.468626Z","created":"2024-02-07T12:18:24.468626Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-07T12:20:10.644 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 1 2024-02-07T12:20:10.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 podman[29956]: 2024-02-07 12:20:10.465539382 +0000 UTC m=+0.019432001 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T12:20:10.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 podman[29956]: 2024-02-07 12:20:10.567806659 +0000 UTC m=+0.121699254 container create c30ccd326a7c82ed6155ff4d4dcec8c896bb356789c27e2e9b3ae37f17c41fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-c, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T12:20:10.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 podman[29956]: 2024-02-07 12:20:10.871759656 +0000 UTC m=+0.425652251 container init c30ccd326a7c82ed6155ff4d4dcec8c896bb356789c27e2e9b3ae37f17c41fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-c, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0) 2024-02-07T12:20:11.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:10 smithi078 ceph-mon[25576]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:10 smithi078 ceph-mon[25576]: from='client.? 172.21.15.171:0/3822172083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T12:20:11.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 podman[29956]: 2024-02-07 12:20:10.877155578 +0000 UTC m=+0.431048173 container start c30ccd326a7c82ed6155ff4d4dcec8c896bb356789c27e2e9b3ae37f17c41fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-07T12:20:11.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: set uid:gid to 167:167 (ceph:ceph) 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev), process ceph-mon, pid 7 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pidfile_write: ignore empty --pid-file 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: load: jerasure load: lrc 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: RocksDB version: 7.9.2 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Git sha 0 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Compile date 2024-02-05 19:41:51 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: DB SUMMARY 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: DB Session ID: PL636LHI3DM10B63CU4P 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: CURRENT file: CURRENT 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: IDENTITY file: IDENTITY 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-07T12:20:11.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.error_if_exists: 0 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.create_if_missing: 0 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.paranoid_checks: 1 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.env: 0x55ffd51efae0 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.fs: PosixFileSystem 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.info_log: 0x55ffd78c18c0 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_file_opening_threads: 16 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.statistics: (nil) 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.use_fsync: 0 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_log_file_size: 0 2024-02-07T12:20:11.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-07T12:20:11.165 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-07T12:20:11.165 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.keep_log_file_num: 1000 2024-02-07T12:20:11.165 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.recycle_log_file_num: 0 2024-02-07T12:20:11.165 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.allow_fallocate: 1 2024-02-07T12:20:11.166 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.allow_mmap_reads: 0 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.allow_mmap_writes: 0 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.use_direct_reads: 0 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.create_missing_column_families: 0 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.db_log_dir: 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.wal_dir: 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.advise_random_on_open: 1 2024-02-07T12:20:11.167 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.db_write_buffer_size: 0 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.write_buffer_manager: 0x55ffd7895400 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.rate_limiter: (nil) 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.wal_recovery_mode: 2 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.enable_thread_tracking: 0 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.enable_pipelined_write: 0 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.unordered_write: 0 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-07T12:20:11.168 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.row_cache: None 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.wal_filter: None 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.allow_ingest_behind: 0 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.two_write_queues: 0 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.manual_wal_flush: 0 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.wal_compression: 0 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.atomic_flush: 0 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-07T12:20:11.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.log_readahead_size: 0 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.best_efforts_recovery: 0 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.allow_data_in_errors: 0 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.db_host_id: __hostname__ 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_background_jobs: 2 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_background_compactions: -1 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_subcompactions: 1 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-07T12:20:11.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_total_wal_size: 0 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_open_files: -1 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bytes_per_sync: 0 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_readahead_size: 0 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_background_flushes: -1 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Compression algorithms supported: 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: kZSTD supported: 0 2024-02-07T12:20:11.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: kXpressCompression supported: 0 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: kLZ4HCCompression supported: 1 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: kZlibCompression supported: 1 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: kSnappyCompression supported: 1 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: kLZ4Compression supported: 1 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: kBZip2Compression supported: 0 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.merge_operator: 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_filter: None 2024-02-07T12:20:11.172 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_filter_factory: None 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.sst_partitioner_factory: None 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ffd78c19c0) 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks: 1 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_top_level_index_and_filter: 1 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_type: 0 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_index_type: 0 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_shortening: 1 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: checksum: 4 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: no_block_cache: 0 2024-02-07T12:20:11.173 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache: 0x55ffd78c5090 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_name: BinnedLRUCache 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_options: 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: capacity : 536870912 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_shard_bits : 4 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: strict_capacity_limit : 0 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: high_pri_pool_ratio: 0.000 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_compressed: (nil) 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: persistent_cache: (nil) 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size: 4096 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size_deviation: 10 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_restart_interval: 16 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_block_restart_interval: 1 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: metadata_block_size: 4096 2024-02-07T12:20:11.174 INFO:journalctl@ceph.mon.c.smithi171.stdout: partition_filters: 0 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: use_delta_encoding: 1 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: filter_policy: bloomfilter 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: whole_key_filtering: 1 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: verify_compression: 0 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: read_amp_bytes_per_bit: 0 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: format_version: 5 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_index_compression: 1 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_align: 0 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: max_auto_readahead_size: 262144 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: prepopulate_block_cache: 0 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: initial_auto_readahead_size: 8192 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_file_reads_for_auto_readahead: 2 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.write_buffer_size: 33554432 2024-02-07T12:20:11.175 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_write_buffer_number: 2 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression: NoCompression 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression: Disabled 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.prefix_extractor: nullptr 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.num_levels: 7 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-07T12:20:11.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression_opts.level: 32767 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression_opts.strategy: 0 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression_opts.enabled: false 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-07T12:20:11.177 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.target_file_size_base: 67108864 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-07T12:20:11.178 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.arena_block_size: 1048576 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.disable_auto_compactions: 0 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-07T12:20:11.179 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.table_properties_collectors: 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.inplace_update_support: 0 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.bloom_locality: 0 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.max_successive_merges: 0 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.paranoid_file_checks: 0 2024-02-07T12:20:11.180 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.force_consistency_checks: 1 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.report_bg_io_stats: 0 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.ttl: 2592000 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.enable_blob_files: false 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.min_blob_size: 0 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.blob_file_size: 268435456 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-07T12:20:11.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.blob_file_starting_level: 0 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c95a07f7-5b8e-4cc3-8678-08a970fd61db 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707308410907206, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707308410907645, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707308410, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c95a07f7-5b8e-4cc3-8678-08a970fd61db", "db_session_id": "PL636LHI3DM10B63CU4P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707308410907725, "job": 1, "event": "recovery_finished"} 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ffd79b6000 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: DB pointer 0x55ffd79a0000 2024-02-07T12:20:11.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** DB Stats ** 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-07T12:20:11.183 INFO:journalctl@ceph.mon.c.smithi171.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-07T12:20:11.184 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache BinnedLRUCache@0x55ffd78c5090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: using public_addr v2:172.21.15.171:0/0 -> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: starting mon.c rank -1 at public addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] at bind addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:20:11.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(???) e0 preinit fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).mds e1 new map 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout: e1 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout: legacy client fscid: -1 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout: No filesystems configured 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-07T12:20:11.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:11.187 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Updating smithi106:/etc/ceph/ceph.conf 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Deploying cephadm binary to smithi171 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Added host smithi171 2024-02-07T12:20:11.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/13457377' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/13457377' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:20:11.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Updating smithi171:/etc/ceph/ceph.conf 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:11.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi106:172.21.15.106=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi106:172.21.15.106=b;smithi171:172.21.15.171=c;count:3 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: Deploying daemon mon.c on smithi171 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: from='client.? 172.21.15.171:0/3822172083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 ceph-mon[30005]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 bash[29956]: c30ccd326a7c82ed6155ff4d4dcec8c896bb356789c27e2e9b3ae37f17c41fcc 2024-02-07T12:20:11.191 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:10 smithi171 systemd[1]: Started Ceph mon.c for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:20:12.471 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-07T12:20:12.472 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-07T12:20:12.718 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:20:16.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: mon.a calling monitor election 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: mon.c calling monitor election 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T12:20:16.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: monmap e2: 2 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:20:16.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: fsmap 2024-02-07T12:20:16.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:16.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: mgrmap e13: y(active, since 50s) 2024-02-07T12:20:16.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: overall HEALTH_OK 2024-02-07T12:20:16.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:16.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:16.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:16.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:16.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:16 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:16.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:20:16.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: mon.a calling monitor election 2024-02-07T12:20:16.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:16.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: mon.c calling monitor election 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: monmap e2: 2 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: fsmap 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: mgrmap e13: y(active, since 50s) 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: overall HEALTH_OK 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:16.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:16.414 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:16 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:17.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:17 smithi078 ceph-mon[25576]: Deploying daemon mon.b on smithi106 2024-02-07T12:20:17.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:17 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:17.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:17 smithi171 ceph-mon[30005]: Deploying daemon mon.b on smithi106 2024-02-07T12:20:17.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:17 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:17.419 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:20:17.419 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"dc35b7c8-c5b2-11ee-95b6-87774f69a715","modified":"2024-02-07T12:20:10.930567Z","created":"2024-02-07T12:18:24.468626Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-07T12:20:17.421 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-02-07T12:20:18.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:18 smithi078 ceph-mon[25576]: from='client.? 172.21.15.171:0/1785483710' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T12:20:18.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:18 smithi078 ceph-mon[25576]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:18.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:18 smithi171 ceph-mon[30005]: from='client.? 172.21.15.171:0/1785483710' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T12:20:18.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:18 smithi171 ceph-mon[30005]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:19.028 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-07T12:20:19.028 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-07T12:20:19.272 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:20:19.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 podman[29655]: 2024-02-07 12:20:19.631300258 +0000 UTC m=+0.017445120 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T12:20:20.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 podman[29655]: 2024-02-07 12:20:19.899179853 +0000 UTC m=+0.285324707 container init 78e5de5d944227790361f61596cf3f23f26ad130f31612d06a68b87ffd0e8b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-b, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-07T12:20:20.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 podman[29655]: 2024-02-07 12:20:19.904193839 +0000 UTC m=+0.290338695 container start 78e5de5d944227790361f61596cf3f23f26ad130f31612d06a68b87ffd0e8b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-07T12:20:20.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: set uid:gid to 167:167 (ceph:ceph) 2024-02-07T12:20:20.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev), process ceph-mon, pid 7 2024-02-07T12:20:20.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pidfile_write: ignore empty --pid-file 2024-02-07T12:20:20.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: load: jerasure load: lrc 2024-02-07T12:20:20.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: RocksDB version: 7.9.2 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Git sha 0 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Compile date 2024-02-05 19:41:51 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: DB SUMMARY 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: DB Session ID: T73QIQM3C1E53POE0QRZ 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: CURRENT file: CURRENT 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: IDENTITY file: IDENTITY 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.error_if_exists: 0 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.create_if_missing: 0 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.paranoid_checks: 1 2024-02-07T12:20:20.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-07T12:20:20.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-07T12:20:20.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-07T12:20:20.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.env: 0x560b264e8ae0 2024-02-07T12:20:20.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.fs: PosixFileSystem 2024-02-07T12:20:20.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.info_log: 0x560b28ae98c0 2024-02-07T12:20:20.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_file_opening_threads: 16 2024-02-07T12:20:20.269 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.statistics: (nil) 2024-02-07T12:20:20.269 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.use_fsync: 0 2024-02-07T12:20:20.269 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_log_file_size: 0 2024-02-07T12:20:20.269 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.keep_log_file_num: 1000 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.recycle_log_file_num: 0 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.allow_fallocate: 1 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.allow_mmap_reads: 0 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.allow_mmap_writes: 0 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.use_direct_reads: 0 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.create_missing_column_families: 0 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.db_log_dir: 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.wal_dir: 2024-02-07T12:20:20.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.advise_random_on_open: 1 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.db_write_buffer_size: 0 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.write_buffer_manager: 0x560b28abd400 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.rate_limiter: (nil) 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-07T12:20:20.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.wal_recovery_mode: 2 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.enable_thread_tracking: 0 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.enable_pipelined_write: 0 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.unordered_write: 0 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.row_cache: None 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.wal_filter: None 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.allow_ingest_behind: 0 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.two_write_queues: 0 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.manual_wal_flush: 0 2024-02-07T12:20:20.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.wal_compression: 0 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.atomic_flush: 0 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.log_readahead_size: 0 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.best_efforts_recovery: 0 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.allow_data_in_errors: 0 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.db_host_id: __hostname__ 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_background_jobs: 2 2024-02-07T12:20:20.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_background_compactions: -1 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_subcompactions: 1 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_total_wal_size: 0 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_open_files: -1 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bytes_per_sync: 0 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_readahead_size: 0 2024-02-07T12:20:20.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_background_flushes: -1 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Compression algorithms supported: 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: kZSTD supported: 0 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: kXpressCompression supported: 0 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: kLZ4HCCompression supported: 1 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: kZlibCompression supported: 1 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: kSnappyCompression supported: 1 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: kLZ4Compression supported: 1 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: kBZip2Compression supported: 0 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-07T12:20:20.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.merge_operator: 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_filter: None 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_filter_factory: None 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.sst_partitioner_factory: None 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560b28ae99c0) 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout: cache_index_and_filter_blocks: 1 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout: pin_top_level_index_and_filter: 1 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout: index_type: 0 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout: data_block_index_type: 0 2024-02-07T12:20:20.276 INFO:journalctl@ceph.mon.b.smithi106.stdout: index_shortening: 1 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: checksum: 4 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: no_block_cache: 0 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_cache: 0x560b28aed090 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_cache_name: BinnedLRUCache 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_cache_options: 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: capacity : 536870912 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: num_shard_bits : 4 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: strict_capacity_limit : 0 2024-02-07T12:20:20.277 INFO:journalctl@ceph.mon.b.smithi106.stdout: high_pri_pool_ratio: 0.000 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_cache_compressed: (nil) 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: persistent_cache: (nil) 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_size: 4096 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_size_deviation: 10 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_restart_interval: 16 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: index_block_restart_interval: 1 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: metadata_block_size: 4096 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: partition_filters: 0 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: use_delta_encoding: 1 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: filter_policy: bloomfilter 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: whole_key_filtering: 1 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: verify_compression: 0 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: read_amp_bytes_per_bit: 0 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: format_version: 5 2024-02-07T12:20:20.278 INFO:journalctl@ceph.mon.b.smithi106.stdout: enable_index_compression: 1 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_align: 0 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout: max_auto_readahead_size: 262144 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout: prepopulate_block_cache: 0 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout: initial_auto_readahead_size: 8192 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout: num_file_reads_for_auto_readahead: 2 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.write_buffer_size: 33554432 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_write_buffer_number: 2 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression: NoCompression 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression: Disabled 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.prefix_extractor: nullptr 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.num_levels: 7 2024-02-07T12:20:20.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression_opts.level: 32767 2024-02-07T12:20:20.280 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression_opts.strategy: 0 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression_opts.enabled: false 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.target_file_size_base: 67108864 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-07T12:20:20.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.arena_block_size: 1048576 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.disable_auto_compactions: 0 2024-02-07T12:20:20.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.table_properties_collectors: 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.inplace_update_support: 0 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-07T12:20:20.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.bloom_locality: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.max_successive_merges: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.paranoid_file_checks: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.force_consistency_checks: 1 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.report_bg_io_stats: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.ttl: 2592000 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.enable_blob_files: false 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.min_blob_size: 0 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.blob_file_size: 268435456 2024-02-07T12:20:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.blob_file_starting_level: 0 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0f3912cc-5dd7-4f59-a9ef-ebca093a476b 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707308419933808, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707308419934206, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707308419, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0f3912cc-5dd7-4f59-a9ef-ebca093a476b", "db_session_id": "T73QIQM3C1E53POE0QRZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707308419934283, "job": 1, "event": "recovery_finished"} 2024-02-07T12:20:20.285 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560b28bde000 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: DB pointer 0x560b28bc8000 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout: ** DB Stats ** 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T12:20:20.286 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: ** Compaction Stats [default] ** 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: ** Compaction Stats [default] ** 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-07T12:20:20.287 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: Cumulative compaction: 0.00 GB write, 0.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: Interval compaction: 0.00 GB write, 0.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: Block cache BinnedLRUCache@0x560b28aed090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-07T12:20:20.288 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: using public_addr v2:172.21.15.106:0/0 -> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: starting mon.b rank -1 at public addrs [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] at bind addrs [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(???) e0 preinit fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).mds e1 new map 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout: e1 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout: legacy client fscid: -1 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout: No filesystems configured 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-07T12:20:20.289 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:20:20.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:20.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Updating smithi106:/etc/ceph/ceph.conf 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Deploying cephadm binary to smithi171 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Added host smithi171 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/13457377' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/13457377' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.293 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Updating smithi171:/etc/ceph/ceph.conf 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:20.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi106:172.21.15.106=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi106:172.21.15.106=b;smithi171:172.21.15.171=c;count:3 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Deploying daemon mon.c on smithi171 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='client.? 172.21.15.171:0/3822172083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.a calling monitor election 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:20.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.c calling monitor election 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: monmap e2: 2 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: fsmap 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mgrmap e13: y(active, since 50s) 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: overall HEALTH_OK 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: Deploying daemon mon.b on smithi106 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: from='client.? 172.21.15.171:0/1785483710' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 ceph-mon[29693]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 bash[29655]: 78e5de5d944227790361f61596cf3f23f26ad130f31612d06a68b87ffd0e8b23 2024-02-07T12:20:20.297 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:19 smithi106 systemd[1]: Started Ceph mon.b for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:20:25.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:25.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: mon.a calling monitor election 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: mon.c calling monitor election 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T12:20:25.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: monmap e3: 3 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:20:25.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: fsmap 2024-02-07T12:20:25.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:25.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: mgrmap e13: y(active, since 59s) 2024-02-07T12:20:25.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-07T12:20:25.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-07T12:20:25.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T12:20:25.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: mon.b (rank 2) addr [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] is down (out of quorum) 2024-02-07T12:20:25.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:25.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:25.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:25.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:20:25.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: mon.a calling monitor election 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: mon.c calling monitor election 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: monmap e3: 3 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: fsmap 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: mgrmap e13: y(active, since 59s) 2024-02-07T12:20:25.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-07T12:20:25.414 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-07T12:20:25.414 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T12:20:25.414 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: mon.b (rank 2) addr [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] is down (out of quorum) 2024-02-07T12:20:25.414 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:25.414 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-02-07T12:20:25.460 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":3,"fsid":"dc35b7c8-c5b2-11ee-95b6-87774f69a715","modified":"2024-02-07T12:20:19.957054Z","created":"2024-02-07T12:18:24.468626Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-07T12:20:25.462 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 3 2024-02-07T12:20:26.050 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-07T12:20:26.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph config generate-minimal-conf 2024-02-07T12:20:26.298 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:20:26.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:26.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:26.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:26.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:26 smithi171 ceph-mon[30005]: from='client.? 172.21.15.171:0/2108473923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T12:20:26.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:26 smithi171 ceph-mon[30005]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:26.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:26.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:26.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:26.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:26.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:26.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:26 smithi078 ceph-mon[25576]: from='client.? 172.21.15.171:0/2108473923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T12:20:26.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:26 smithi078 ceph-mon[25576]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:26.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:26.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:27.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:27.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:20:27.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:27.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: mon.a calling monitor election 2024-02-07T12:20:27.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: mon.c calling monitor election 2024-02-07T12:20:27.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:20:27.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:27.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:27.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: monmap e3: 3 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: fsmap 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: mgrmap e13: y(active, since 59s) 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: mon.b (rank 2) addr [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] is down (out of quorum) 2024-02-07T12:20:27.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:27.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:27.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:27.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:27.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:27.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='client.? 172.21.15.171:0/2108473923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T12:20:27.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:27.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:27.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:28.118 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: mon.b calling monitor election 2024-02-07T12:20:28.118 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: mon.b calling monitor election 2024-02-07T12:20:28.118 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: mon.c calling monitor election 2024-02-07T12:20:28.118 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: mon.a calling monitor election 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: monmap e3: 3 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: fsmap 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: mgrmap e13: y(active, since 61s) 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: Cluster is now healthy 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: overall HEALTH_OK 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.119 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:28.120 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T12:20:28.121 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:28.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: mon.b calling monitor election 2024-02-07T12:20:28.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: mon.b calling monitor election 2024-02-07T12:20:28.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: mon.c calling monitor election 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: mon.a calling monitor election 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: monmap e3: 3 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: fsmap 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: mgrmap e13: y(active, since 61s) 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: Cluster is now healthy 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: overall HEALTH_OK 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T12:20:28.164 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:28.199 INFO:teuthology.orchestra.run.smithi078.stdout:# minimal ceph.conf for dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:20:28.199 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2024-02-07T12:20:28.199 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:20:28.199 INFO:teuthology.orchestra.run.smithi078.stdout: mon_host = [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-02-07T12:20:28.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: mon.b calling monitor election 2024-02-07T12:20:28.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: mon.b calling monitor election 2024-02-07T12:20:28.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: mon.c calling monitor election 2024-02-07T12:20:28.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: mon.a calling monitor election 2024-02-07T12:20:28.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-07T12:20:28.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: monmap e3: 3 mons at {a=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T12:20:28.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: fsmap 2024-02-07T12:20:28.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: mgrmap e13: y(active, since 61s) 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: Cluster is now healthy 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: overall HEALTH_OK 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:28.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:28.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T12:20:28.268 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:28.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:28 smithi078 ceph-mon[25576]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:28.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:28 smithi078 ceph-mon[25576]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:28.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:28 smithi078 ceph-mon[25576]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:28.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:28 smithi078 ceph-mon[25576]: Reconfiguring mon.a (unknown last config time)... 2024-02-07T12:20:28.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:28 smithi078 ceph-mon[25576]: Reconfiguring daemon mon.a on smithi078 2024-02-07T12:20:28.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:28 smithi078 ceph-mon[25576]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:28.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:28 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:28.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:28 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3511329351' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:29.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:28 smithi171 ceph-mon[30005]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:29.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:28 smithi171 ceph-mon[30005]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:29.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:28 smithi171 ceph-mon[30005]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:29.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:28 smithi171 ceph-mon[30005]: Reconfiguring mon.a (unknown last config time)... 2024-02-07T12:20:29.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:28 smithi171 ceph-mon[30005]: Reconfiguring daemon mon.a on smithi078 2024-02-07T12:20:29.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:28 smithi171 ceph-mon[30005]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:29.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:28 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:29.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:28 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/3511329351' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:29.247 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-07T12:20:29.247 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:20:29.247 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-02-07T12:20:29.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:28 smithi106 ceph-mon[29693]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:29.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:28 smithi106 ceph-mon[29693]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:29.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:28 smithi106 ceph-mon[29693]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:20:29.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:28 smithi106 ceph-mon[29693]: Reconfiguring mon.a (unknown last config time)... 2024-02-07T12:20:29.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:28 smithi106 ceph-mon[29693]: Reconfiguring daemon mon.a on smithi078 2024-02-07T12:20:29.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:28 smithi106 ceph-mon[29693]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:29.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:28 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:20:29.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:28 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/3511329351' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:29.282 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:20:29.282 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:20:29.355 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:20:29.356 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.conf 2024-02-07T12:20:29.391 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:20:29.391 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:20:29.461 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:20:29.462 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-02-07T12:20:29.496 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:20:29.497 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:20:29.568 INFO:tasks.cephadm:Adding mgr.y on smithi078 2024-02-07T12:20:29.568 INFO:tasks.cephadm:Adding mgr.x on smithi106 2024-02-07T12:20:29.568 INFO:tasks.cephadm:Adding mgr.z on smithi171 2024-02-07T12:20:29.568 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch apply mgr '3;smithi078=y;smithi106=x;smithi171=z' 2024-02-07T12:20:29.845 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:20:30.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:30 smithi171 ceph-mon[30005]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:30.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:30 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:30.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:30 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:30.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:30 smithi171 ceph-mon[30005]: Reconfiguring mon.b (monmap changed)... 2024-02-07T12:20:30.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:30 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:30.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:30 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T12:20:30.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:30 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:30.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:30 smithi171 ceph-mon[30005]: Reconfiguring daemon mon.b on smithi106 2024-02-07T12:20:31.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:30 smithi106 ceph-mon[29693]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:31.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:30 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:31.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:30 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:31.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:30 smithi106 ceph-mon[29693]: Reconfiguring mon.b (monmap changed)... 2024-02-07T12:20:31.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:30 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:31.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:30 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T12:20:31.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:30 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:31.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:30 smithi106 ceph-mon[29693]: Reconfiguring daemon mon.b on smithi106 2024-02-07T12:20:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:30 smithi078 ceph-mon[25576]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:30 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:30 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:30 smithi078 ceph-mon[25576]: Reconfiguring mon.b (monmap changed)... 2024-02-07T12:20:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:30 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:30 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T12:20:31.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:30 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:31.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:30 smithi078 ceph-mon[25576]: Reconfiguring daemon mon.b on smithi106 2024-02-07T12:20:31.602 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mgr update... 2024-02-07T12:20:32.157 DEBUG:teuthology.orchestra.run.smithi106:mgr.x> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.x.service 2024-02-07T12:20:32.160 DEBUG:teuthology.orchestra.run.smithi171:mgr.z> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.z.service 2024-02-07T12:20:32.162 INFO:tasks.cephadm:Deploying OSDs... 2024-02-07T12:20:32.162 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:20:32.163 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T12:20:32.182 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T12:20:32.182 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/nvme1n1 2024-02-07T12:20:32.240 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/nvme1n1 2024-02-07T12:20:32.241 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:32.241 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 763 Links: 1 Device type: 103,2 2024-02-07T12:20:32.241 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:32.241 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:32.241 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-07 12:19:36.160366688 +0000 2024-02-07T12:20:32.241 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-07 12:17:24.463373640 +0000 2024-02-07T12:20:32.241 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-07 12:17:24.463373640 +0000 2024-02-07T12:20:32.241 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-07 12:17:24.453373803 +0000 2024-02-07T12:20:32.241 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-07T12:20:32.312 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-07T12:20:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-07T12:20:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000259544 s, 2.0 MB/s 2024-02-07T12:20:32.314 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-07T12:20:32.373 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/nvme2n1 2024-02-07T12:20:32.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:32.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:32.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: Reconfiguring mon.c (monmap changed)... 2024-02-07T12:20:32.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:32.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T12:20:32.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:32.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: Reconfiguring daemon mon.c on smithi171 2024-02-07T12:20:32.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=y;smithi106=x;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:32.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: Saving service mgr spec with placement smithi078=y;smithi106=x;smithi171=z;count:3 2024-02-07T12:20:32.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:32.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:32 smithi171 ceph-mon[30005]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:32.430 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/nvme2n1 2024-02-07T12:20:32.431 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:32.431 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-07T12:20:32.431 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:32.431 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:32.431 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-07 12:19:36.172366493 +0000 2024-02-07T12:20:32.431 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-07 12:17:24.624371015 +0000 2024-02-07T12:20:32.431 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-07 12:17:24.624371015 +0000 2024-02-07T12:20:32.431 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-07 12:17:24.616371146 +0000 2024-02-07T12:20:32.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-07T12:20:32.460 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: Reconfiguring mon.c (monmap changed)... 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: Reconfiguring daemon mon.c on smithi171 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=y;smithi106=x;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: Saving service mgr spec with placement smithi078=y;smithi106=x;smithi171=z;count:3 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:32.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:32 smithi106 ceph-mon[29693]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-07T12:20:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-07T12:20:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000260214 s, 2.0 MB/s 2024-02-07T12:20:32.501 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-07T12:20:32.558 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/nvme3n1 2024-02-07T12:20:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/nvme3n1 2024-02-07T12:20:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:32.615 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-07T12:20:32.615 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:32.615 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:32.615 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-07 12:19:36.182366330 +0000 2024-02-07T12:20:32.615 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-07 12:17:24.774368570 +0000 2024-02-07T12:20:32.615 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-07 12:17:24.774368570 +0000 2024-02-07T12:20:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-07 12:17:24.763368749 +0000 2024-02-07T12:20:32.616 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-07T12:20:32.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:32.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:32.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: Reconfiguring mon.c (monmap changed)... 2024-02-07T12:20:32.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T12:20:32.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T12:20:32.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:32.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: Reconfiguring daemon mon.c on smithi171 2024-02-07T12:20:32.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=y;smithi106=x;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:32.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: Saving service mgr spec with placement smithi078=y;smithi106=x;smithi171=z;count:3 2024-02-07T12:20:32.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:32.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:32 smithi078 ceph-mon[25576]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:32.658 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-07T12:20:32.658 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-07T12:20:32.658 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000196843 s, 2.6 MB/s 2024-02-07T12:20:32.659 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-07T12:20:32.716 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/nvme4n1 2024-02-07T12:20:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/nvme4n1 2024-02-07T12:20:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:32.772 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-07T12:20:32.772 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:32.772 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:32.772 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-07 12:19:36.193366150 +0000 2024-02-07T12:20:32.772 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-07 12:17:24.921366173 +0000 2024-02-07T12:20:32.773 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-07 12:17:24.921366173 +0000 2024-02-07T12:20:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-07 12:17:24.909366369 +0000 2024-02-07T12:20:32.773 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-07T12:20:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-07T12:20:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-07T12:20:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000218124 s, 2.3 MB/s 2024-02-07T12:20:32.839 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-07T12:20:32.896 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:20:32.897 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T12:20:32.915 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T12:20:32.916 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/nvme1n1 2024-02-07T12:20:32.974 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/nvme1n1 2024-02-07T12:20:32.974 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:32.975 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 764 Links: 1 Device type: 103,2 2024-02-07T12:20:32.975 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:32.975 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:32.975 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-02-07 12:19:53.066332050 +0000 2024-02-07T12:20:32.975 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-02-07 12:17:26.116942630 +0000 2024-02-07T12:20:32.975 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-02-07 12:17:26.116942630 +0000 2024-02-07T12:20:32.975 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-02-07 12:17:26.108944299 +0000 2024-02-07T12:20:32.975 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-07T12:20:33.044 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-02-07T12:20:33.045 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-02-07T12:20:33.045 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000282562 s, 1.8 MB/s 2024-02-07T12:20:33.046 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-07T12:20:33.105 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/nvme2n1 2024-02-07T12:20:33.161 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/nvme2n1 2024-02-07T12:20:33.161 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:33.162 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-07T12:20:33.162 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:33.162 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:33.162 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-02-07 12:19:53.075331836 +0000 2024-02-07T12:20:33.162 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-02-07 12:17:26.284907568 +0000 2024-02-07T12:20:33.162 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-02-07 12:17:26.284907568 +0000 2024-02-07T12:20:33.162 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-02-07 12:17:26.275909446 +0000 2024-02-07T12:20:33.162 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-07T12:20:33.230 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-02-07T12:20:33.230 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-02-07T12:20:33.230 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000295128 s, 1.7 MB/s 2024-02-07T12:20:33.232 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-07T12:20:33.292 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/nvme3n1 2024-02-07T12:20:33.348 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/nvme3n1 2024-02-07T12:20:33.348 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:33.348 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 788 Links: 1 Device type: 103,6 2024-02-07T12:20:33.348 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:33.349 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:33.349 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-02-07 12:19:53.085331600 +0000 2024-02-07T12:20:33.349 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-02-07 12:17:26.397883985 +0000 2024-02-07T12:20:33.349 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-02-07 12:17:26.397883985 +0000 2024-02-07T12:20:33.349 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-02-07 12:17:26.390885446 +0000 2024-02-07T12:20:33.349 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-07T12:20:33.416 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-02-07T12:20:33.416 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-02-07T12:20:33.417 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000219214 s, 2.3 MB/s 2024-02-07T12:20:33.418 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-07T12:20:33.476 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/nvme4n1 2024-02-07T12:20:33.532 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/nvme4n1 2024-02-07T12:20:33.532 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:33.532 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 798 Links: 1 Device type: 103,8 2024-02-07T12:20:33.532 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:33.533 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:33.533 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-02-07 12:19:53.095331363 +0000 2024-02-07T12:20:33.533 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-02-07 12:17:26.547852680 +0000 2024-02-07T12:20:33.533 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-02-07 12:17:26.547852680 +0000 2024-02-07T12:20:33.533 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-02-07 12:17:26.537854767 +0000 2024-02-07T12:20:33.533 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-07T12:20:33.600 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-02-07T12:20:33.600 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-02-07T12:20:33.600 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000244482 s, 2.1 MB/s 2024-02-07T12:20:33.601 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-07T12:20:33.657 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:20:33.657 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T12:20:33.673 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T12:20:33.674 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/nvme1n1 2024-02-07T12:20:33.729 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/nvme1n1 2024-02-07T12:20:33.729 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:33.729 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 764 Links: 1 Device type: 103,2 2024-02-07T12:20:33.729 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:33.729 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:33.730 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-02-07 12:20:02.668833912 +0000 2024-02-07T12:20:33.730 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-02-07 12:17:27.738737020 +0000 2024-02-07T12:20:33.730 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-02-07 12:17:27.738737020 +0000 2024-02-07T12:20:33.730 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-02-07 12:17:27.730737169 +0000 2024-02-07T12:20:33.730 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-07T12:20:33.797 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-02-07T12:20:33.797 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-02-07T12:20:33.797 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000303153 s, 1.7 MB/s 2024-02-07T12:20:33.798 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-07T12:20:33.855 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/nvme2n1 2024-02-07T12:20:33.911 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/nvme2n1 2024-02-07T12:20:33.911 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:33.911 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-07T12:20:33.911 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:33.911 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:33.912 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-02-07 12:20:02.680833687 +0000 2024-02-07T12:20:33.912 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-02-07 12:17:27.898734018 +0000 2024-02-07T12:20:33.912 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-02-07 12:17:27.898734018 +0000 2024-02-07T12:20:33.912 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-02-07 12:17:27.889734187 +0000 2024-02-07T12:20:33.912 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-07T12:20:33.978 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-02-07T12:20:33.978 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-02-07T12:20:33.978 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000244982 s, 2.1 MB/s 2024-02-07T12:20:33.979 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-07T12:20:34.035 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/nvme3n1 2024-02-07T12:20:34.090 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/nvme3n1 2024-02-07T12:20:34.090 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:34.091 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-07T12:20:34.091 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:34.091 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:34.091 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-02-07 12:20:02.691833482 +0000 2024-02-07T12:20:34.091 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-02-07 12:17:28.011731898 +0000 2024-02-07T12:20:34.091 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-02-07 12:17:28.011731898 +0000 2024-02-07T12:20:34.091 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-02-07 12:17:27.999732123 +0000 2024-02-07T12:20:34.091 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-07T12:20:34.157 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-02-07T12:20:34.157 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-02-07T12:20:34.157 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000262456 s, 2.0 MB/s 2024-02-07T12:20:34.160 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-07T12:20:34.214 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/nvme4n1 2024-02-07T12:20:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/nvme4n1 2024-02-07T12:20:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T12:20:34.270 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-07T12:20:34.270 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T12:20:34.271 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T12:20:34.271 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-02-07 12:20:02.702833276 +0000 2024-02-07T12:20:34.271 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-02-07 12:17:28.158729140 +0000 2024-02-07T12:20:34.271 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-02-07 12:17:28.158729140 +0000 2024-02-07T12:20:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-02-07 12:17:28.146729365 +0000 2024-02-07T12:20:34.271 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-07T12:20:34.337 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-02-07T12:20:34.337 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-02-07T12:20:34.337 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000187836 s, 2.7 MB/s 2024-02-07T12:20:34.338 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-07T12:20:34.394 INFO:tasks.cephadm:Deploying osd.0 on smithi078 with /dev/nvme4n1... 2024-02-07T12:20:34.394 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T12:20:34.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:34.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:34 smithi171 ceph-mon[30005]: Deploying daemon mgr.z on smithi171 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T12:20:34.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:34.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:34 smithi106 ceph-mon[29693]: Deploying daemon mgr.z on smithi171 2024-02-07T12:20:34.627 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T12:20:34.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:34.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:34 smithi078 ceph-mon[25576]: Deploying daemon mgr.z on smithi171 2024-02-07T12:20:36.099 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:35 smithi171 podman[32062]: 2024-02-07 12:20:35.883758586 +0000 UTC m=+0.018428774 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T12:20:36.100 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:36 smithi171 podman[32062]: 2024-02-07 12:20:36.097487052 +0000 UTC m=+0.232157244 container create 81a09960593613277f21a9f1e4e780803328659335936c16e55bdf9de27e94a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T12:20:36.377 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:36 smithi171 podman[32062]: 2024-02-07 12:20:36.300132771 +0000 UTC m=+0.434802960 container init 81a09960593613277f21a9f1e4e780803328659335936c16e55bdf9de27e94a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True) 2024-02-07T12:20:36.377 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:36 smithi171 podman[32062]: 2024-02-07 12:20:36.305745604 +0000 UTC m=+0.440415794 container start 81a09960593613277f21a9f1e4e780803328659335936c16e55bdf9de27e94a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-07T12:20:36.377 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-mgr[32102]: -- 172.21.15.171:0/149215195 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558afe533520 con 0x558afe508c00 2024-02-07T12:20:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/nvme4n1 2024-02-07T12:20:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-07T12:20:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-07T12:20:36.444 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-02-07T12:20:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-02-07T12:20:36.444 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132245 s, 793 MB/s 2024-02-07T12:20:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-02-07T12:20:36.631 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:36 smithi171 bash[32062]: 81a09960593613277f21a9f1e4e780803328659335936c16e55bdf9de27e94a0 2024-02-07T12:20:36.631 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:36 smithi171 systemd[1]: Started Ceph mgr.z for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:20:36.912 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:36.868+0000 7f17b2633200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T12:20:36.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-mon[30005]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:36.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:36.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:36.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:36.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T12:20:36.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T12:20:36.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T12:20:36.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:37.062 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:36 smithi078 ceph-mon[25576]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:37.062 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:36 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:37.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:36 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:37.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:36 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:37.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:36 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T12:20:37.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:36 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T12:20:37.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:36 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T12:20:37.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:36 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:37.090 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi078:/dev/nvme4n1 2024-02-07T12:20:37.187 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:36 smithi106 ceph-mon[29693]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:37.188 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:36 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:37.188 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:36 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:37.188 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:36 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:37.188 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:36 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T12:20:37.188 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:36 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T12:20:37.188 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:36 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T12:20:37.188 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:36 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:20:37.378 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:36 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:36.962+0000 7f17b2633200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T12:20:37.662 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:37 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:37.604+0000 7f17b2633200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T12:20:37.999 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:37 smithi106 ceph-mon[29693]: Deploying daemon mgr.x on smithi106 2024-02-07T12:20:38.015 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:37 smithi171 ceph-mon[30005]: Deploying daemon mgr.x on smithi106 2024-02-07T12:20:38.015 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:37 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:37.696+0000 7f17b2633200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T12:20:38.015 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:37 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:37.837+0000 7f17b2633200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T12:20:38.015 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:37 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:37.931+0000 7f17b2633200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T12:20:38.015 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:38 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:38.013+0000 7f17b2633200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T12:20:38.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:37 smithi078 ceph-mon[25576]: Deploying daemon mgr.x on smithi106 2024-02-07T12:20:38.563 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:38 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:38.274+0000 7f17b2633200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T12:20:38.563 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:38 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:38.383+0000 7f17b2633200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T12:20:38.564 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:38 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:38.468+0000 7f17b2633200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T12:20:38.564 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:38 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:38.561+0000 7f17b2633200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T12:20:38.665 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:38 smithi106 systemd[1]: Starting Ceph mgr.x for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:20:38.840 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:38 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:38.737+0000 7f17b2633200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T12:20:38.916 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:38 smithi106 ceph-mon[29693]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:38.916 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:38 smithi106 podman[31137]: 2024-02-07 12:20:38.566333366 +0000 UTC m=+0.017347845 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T12:20:38.916 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:38 smithi106 podman[31137]: 2024-02-07 12:20:38.675211673 +0000 UTC m=+0.126226193 container create 3deecab659e4a0bba5b9c1afa811be0b44abd670736b15ef42cac9c42bf624ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-07T12:20:38.916 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:38 smithi106 podman[31137]: 2024-02-07 12:20:38.884475495 +0000 UTC m=+0.335489974 container init 3deecab659e4a0bba5b9c1afa811be0b44abd670736b15ef42cac9c42bf624ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T12:20:38.916 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:38 smithi106 podman[31137]: 2024-02-07 12:20:38.88991003 +0000 UTC m=+0.340924509 container start 3deecab659e4a0bba5b9c1afa811be0b44abd670736b15ef42cac9c42bf624ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T12:20:38.916 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:38 smithi106 ceph-mgr[31175]: -- 172.21.15.106:0/674836882 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55602c781520 con 0x55602c757000 2024-02-07T12:20:39.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:38 smithi078 ceph-mon[25576]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:39.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:38 smithi171 ceph-mon[30005]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:39.265 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:39 smithi106 bash[31137]: 3deecab659e4a0bba5b9c1afa811be0b44abd670736b15ef42cac9c42bf624ae 2024-02-07T12:20:39.266 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:39 smithi106 systemd[1]: Started Ceph mgr.x for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:20:39.765 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:39 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:39.440+0000 7f3606f0d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T12:20:39.766 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:39 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:39.535+0000 7f3606f0d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T12:20:40.069 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:39 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:39.730+0000 7f17b2633200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T12:20:40.069 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:39 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:39.877+0000 7f17b2633200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T12:20:40.322 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:40.208+0000 7f3606f0d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T12:20:40.322 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:40.302+0000 7f3606f0d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:40.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-mon[29693]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:40.227+0000 7f17b2633200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:40.316+0000 7f17b2633200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T12:20:40.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:40.331 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:40.331 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-mon[30005]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:40.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:40.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:20:40.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T12:20:40.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T12:20:40.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:40.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:40.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:40 smithi078 ceph-mon[25576]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:40.633 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:40.446+0000 7f3606f0d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T12:20:40.633 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:40.541+0000 7f3606f0d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T12:20:40.633 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:40.630+0000 7f3606f0d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T12:20:40.662 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:40.510+0000 7f17b2633200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:20:40.989 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:40.702+0000 7f17b2633200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T12:20:41.183 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:40 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:40.896+0000 7f3606f0d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T12:20:41.184 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:41.004+0000 7f3606f0d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T12:20:41.184 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:41.088+0000 7f3606f0d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T12:20:41.184 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:41.181+0000 7f3606f0d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T12:20:41.248 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:40 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:40.987+0000 7f17b2633200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:20:41.249 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:41.246+0000 7f17b2633200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T12:20:41.515 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:41.355+0000 7f3606f0d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T12:20:41.618 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:41.477+0000 7f17b2633200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T12:20:41.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:41.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:41.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: Reconfiguring mgr.y (unknown last config time)... 2024-02-07T12:20:41.883 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T12:20:41.883 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T12:20:41.883 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:41.883 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: Reconfiguring daemon mgr.y on smithi078 2024-02-07T12:20:41.883 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:41 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: Reconfiguring mgr.y (unknown last config time)... 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T12:20:41.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T12:20:41.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:41.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: Reconfiguring daemon mgr.y on smithi078 2024-02-07T12:20:41.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:41 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:42.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:42.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:42.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:42.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:42.017 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:42.017 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:42.017 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:42.017 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: Reconfiguring mgr.y (unknown last config time)... 2024-02-07T12:20:42.017 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T12:20:42.017 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T12:20:42.018 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:42.018 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: Reconfiguring daemon mgr.y on smithi078 2024-02-07T12:20:42.018 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:41 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:42.622 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:42 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:42.269+0000 7f17b2633200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T12:20:42.623 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:42 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:42.317+0000 7f3606f0d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T12:20:42.623 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:42 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:42.447+0000 7f3606f0d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T12:20:42.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:42 smithi078 ceph-mon[25576]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:42.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:42 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3525867534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0658c29-df47-4181-b6a5-3aac7717f002"}]: dispatch 2024-02-07T12:20:42.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:42 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3525867534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0658c29-df47-4181-b6a5-3aac7717f002"}]': finished 2024-02-07T12:20:42.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:42 smithi078 ceph-mon[25576]: osdmap e5: 1 total, 0 up, 1 in 2024-02-07T12:20:42.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:42 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:20:42.912 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:42 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:42.843+0000 7f17b2633200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T12:20:42.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:42 smithi171 ceph-mon[30005]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:42.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:42 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/3525867534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0658c29-df47-4181-b6a5-3aac7717f002"}]: dispatch 2024-02-07T12:20:42.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:42 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/3525867534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0658c29-df47-4181-b6a5-3aac7717f002"}]': finished 2024-02-07T12:20:42.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:42 smithi171 ceph-mon[30005]: osdmap e5: 1 total, 0 up, 1 in 2024-02-07T12:20:42.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:42 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:20:43.016 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:42 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:42.776+0000 7f3606f0d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T12:20:43.016 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:42 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:42.866+0000 7f3606f0d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T12:20:43.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:42 smithi106 ceph-mon[29693]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:43.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:42 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/3525867534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0658c29-df47-4181-b6a5-3aac7717f002"}]: dispatch 2024-02-07T12:20:43.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:42 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/3525867534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0658c29-df47-4181-b6a5-3aac7717f002"}]': finished 2024-02-07T12:20:43.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:42 smithi106 ceph-mon[29693]: osdmap e5: 1 total, 0 up, 1 in 2024-02-07T12:20:43.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:42 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:20:43.412 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:42 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:42.939+0000 7f17b2633200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T12:20:43.412 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:20:43 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:20:43.151+0000 7f17b2633200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T12:20:43.516 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:43.059+0000 7f3606f0d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:20:43.516 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:43.240+0000 7f3606f0d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T12:20:43.779 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:43.519+0000 7f3606f0d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:20:43.779 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:43.776+0000 7f3606f0d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T12:20:43.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-mon[29693]: Standby manager daemon z started 2024-02-07T12:20:43.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T12:20:43.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:20:43.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T12:20:43.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:20:43.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:43 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/2450675894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T12:20:43.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:43 smithi078 ceph-mon[25576]: Standby manager daemon z started 2024-02-07T12:20:43.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:43 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T12:20:43.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:43 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:20:43.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:43 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T12:20:43.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:43 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:20:43.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:43 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2450675894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T12:20:43.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:43 smithi171 ceph-mon[30005]: Standby manager daemon z started 2024-02-07T12:20:43.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:43 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T12:20:43.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:43 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:20:43.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:43 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T12:20:43.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:43 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.171:0/3637669405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:20:43.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:43 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/2450675894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T12:20:44.265 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:44 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:44.009+0000 7f3606f0d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T12:20:44.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:44 smithi171 ceph-mon[30005]: mgrmap e14: y(active, since 77s), standbys: z 2024-02-07T12:20:44.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:44 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T12:20:44.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:44 smithi171 ceph-mon[30005]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:45.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:44 smithi106 ceph-mon[29693]: mgrmap e14: y(active, since 77s), standbys: z 2024-02-07T12:20:45.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:44 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T12:20:45.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:44 smithi106 ceph-mon[29693]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:45.016 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:44 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:44.793+0000 7f3606f0d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T12:20:45.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:44 smithi078 ceph-mon[25576]: mgrmap e14: y(active, since 77s), standbys: z 2024-02-07T12:20:45.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:44 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T12:20:45.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:44 smithi078 ceph-mon[25576]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:45.655 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:45 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:45.349+0000 7f3606f0d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T12:20:45.655 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:45 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:45.444+0000 7f3606f0d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T12:20:45.656 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:20:45 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:20:45.653+0000 7f3606f0d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T12:20:46.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:45 smithi106 ceph-mon[29693]: Standby manager daemon x started 2024-02-07T12:20:46.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:45 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T12:20:46.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:45 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:20:46.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:45 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T12:20:46.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:45 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:20:46.031 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:45 smithi078 ceph-mon[25576]: Standby manager daemon x started 2024-02-07T12:20:46.031 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:45 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T12:20:46.032 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:45 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:20:46.032 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:45 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T12:20:46.032 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:45 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:20:46.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:45 smithi171 ceph-mon[30005]: Standby manager daemon x started 2024-02-07T12:20:46.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:45 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T12:20:46.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:45 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:20:46.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:45 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T12:20:46.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:45 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.106:0/1700287683' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:20:47.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:46 smithi106 ceph-mon[29693]: mgrmap e15: y(active, since 79s), standbys: z, x 2024-02-07T12:20:47.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:46 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T12:20:47.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:46 smithi106 ceph-mon[29693]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:47.078 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:46 smithi078 ceph-mon[25576]: mgrmap e15: y(active, since 79s), standbys: z, x 2024-02-07T12:20:47.078 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:46 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T12:20:47.079 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:46 smithi078 ceph-mon[25576]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:47.161 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:46 smithi171 ceph-mon[30005]: mgrmap e15: y(active, since 79s), standbys: z, x 2024-02-07T12:20:47.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:46 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T12:20:47.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:46 smithi171 ceph-mon[30005]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:48.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:48.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:48.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:48.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:48.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:48.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:48.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:48 smithi078 ceph-mon[25576]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:48.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:48 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:48.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:48 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:48.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:48 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:48.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:48 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:48.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:48 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:48.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:48 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:48.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:48 smithi171 ceph-mon[30005]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:48.428 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:48 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:48.429 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:48 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:48.429 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:48 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:48.429 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:48 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:48.429 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:48 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:48.429 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:48 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:48.429 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:48 smithi106 ceph-mon[29693]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:50.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:49 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-07T12:20:50.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:49 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:50.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:49 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-07T12:20:50.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:49 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:50.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:49 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-07T12:20:50.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:49 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:50.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:50 smithi078 ceph-mon[25576]: Deploying daemon osd.0 on smithi078 2024-02-07T12:20:50.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:50 smithi078 ceph-mon[25576]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:51.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:50 smithi106 ceph-mon[29693]: Deploying daemon osd.0 on smithi078 2024-02-07T12:20:51.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:50 smithi106 ceph-mon[29693]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:50 smithi171 ceph-mon[30005]: Deploying daemon osd.0 on smithi078 2024-02-07T12:20:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:50 smithi171 ceph-mon[30005]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:53.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:52 smithi078 ceph-mon[25576]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:53.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:52 smithi171 ceph-mon[30005]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:53.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:52 smithi106 ceph-mon[29693]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:55.124 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:54 smithi078 ceph-mon[25576]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:55.161 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:54 smithi171 ceph-mon[30005]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:55.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:54 smithi106 ceph-mon[29693]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:56.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:56 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:56.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:56 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:56.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:56 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:56.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:56 smithi078 ceph-mon[25576]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:56.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:56 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:56.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:56 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:56.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:56 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:56.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:56 smithi171 ceph-mon[30005]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:56.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:56 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:56.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:56 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:56.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:56 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:56.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:56 smithi106 ceph-mon[29693]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:59.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:58 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:58 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:58 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:59.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:58 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:59.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:58 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:58 smithi106 ceph-mon[29693]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:59.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:58 smithi106 ceph-mon[29693]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-07T12:20:59.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:59.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:59.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:58 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:58 smithi078 ceph-mon[25576]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:59.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:58 smithi078 ceph-mon[25576]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-07T12:20:59.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:58 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:58 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:58 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:20:59.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:58 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:20:59.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:58 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:58 smithi171 ceph-mon[30005]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:20:59.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:58 smithi171 ceph-mon[30005]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-07T12:20:59.177 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 0 on host 'smithi078' 2024-02-07T12:20:59.832 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:59 smithi078 ceph-mon[25576]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-07T12:20:59.832 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:59 smithi078 ceph-mon[25576]: osdmap e6: 1 total, 0 up, 1 in 2024-02-07T12:20:59.832 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:59 smithi078 ceph-mon[25576]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-02-07T12:20:59.832 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:59 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:20:59.832 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:59 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.832 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:59 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:20:59.832 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:20:59 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:20:59.859 DEBUG:teuthology.orchestra.run.smithi078:osd.0> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@osd.0.service 2024-02-07T12:20:59.862 INFO:tasks.cephadm:Deploying osd.1 on smithi106 with /dev/nvme4n1... 2024-02-07T12:20:59.862 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-07T12:21:00.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:59 smithi106 ceph-mon[29693]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-07T12:21:00.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:59 smithi106 ceph-mon[29693]: osdmap e6: 1 total, 0 up, 1 in 2024-02-07T12:21:00.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:59 smithi106 ceph-mon[29693]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-02-07T12:21:00.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:59 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:00.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:59 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:00.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:59 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:00.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:20:59 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:00.106 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.b/config 2024-02-07T12:21:00.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:59 smithi171 ceph-mon[30005]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-07T12:21:00.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:59 smithi171 ceph-mon[30005]: osdmap e6: 1 total, 0 up, 1 in 2024-02-07T12:21:00.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:59 smithi171 ceph-mon[30005]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-02-07T12:21:00.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:59 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:00.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:59 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:00.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:59 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:00.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:20:59 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:00.334 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:00 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0[40083]: 2024-02-07T12:21:00.228+0000 7f121db33700 -1 osd.0 0 waiting for initial osdmap 2024-02-07T12:21:00.335 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:00 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0[40083]: 2024-02-07T12:21:00.230+0000 7f1215124700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-07T12:21:01.087 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:00 smithi078 ceph-mon[25576]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-02-07T12:21:01.088 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:00 smithi078 ceph-mon[25576]: osdmap e7: 1 total, 0 up, 1 in 2024-02-07T12:21:01.088 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:00 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:01.088 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:00 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:01.088 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:00 smithi078 ceph-mon[25576]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:21:01.088 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:00 smithi078 ceph-mon[25576]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' 2024-02-07T12:21:01.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:00 smithi171 ceph-mon[30005]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-02-07T12:21:01.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:00 smithi171 ceph-mon[30005]: osdmap e7: 1 total, 0 up, 1 in 2024-02-07T12:21:01.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:00 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:01.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:00 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:01.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:00 smithi171 ceph-mon[30005]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:21:01.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:00 smithi171 ceph-mon[30005]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' 2024-02-07T12:21:01.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:00 smithi106 ceph-mon[29693]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-02-07T12:21:01.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:00 smithi106 ceph-mon[29693]: osdmap e7: 1 total, 0 up, 1 in 2024-02-07T12:21:01.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:00 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:01.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:00 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:01.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:00 smithi106 ceph-mon[29693]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T12:21:01.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:00 smithi106 ceph-mon[29693]: from='osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194]' entity='osd.0' 2024-02-07T12:21:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/nvme4n1 2024-02-07T12:21:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-07T12:21:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-07T12:21:01.977 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-02-07T12:21:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-02-07T12:21:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0139351 s, 752 MB/s 2024-02-07T12:21:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-02-07T12:21:02.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:01 smithi078 ceph-mon[25576]: purged_snaps scrub starts 2024-02-07T12:21:02.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:01 smithi078 ceph-mon[25576]: purged_snaps scrub ok 2024-02-07T12:21:02.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:02.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:01 smithi078 ceph-mon[25576]: osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194] boot 2024-02-07T12:21:02.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:01 smithi078 ceph-mon[25576]: osdmap e8: 1 total, 1 up, 1 in 2024-02-07T12:21:02.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:02.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:01 smithi171 ceph-mon[30005]: purged_snaps scrub starts 2024-02-07T12:21:02.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:01 smithi171 ceph-mon[30005]: purged_snaps scrub ok 2024-02-07T12:21:02.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:02.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:01 smithi171 ceph-mon[30005]: osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194] boot 2024-02-07T12:21:02.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:01 smithi171 ceph-mon[30005]: osdmap e8: 1 total, 1 up, 1 in 2024-02-07T12:21:02.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:01 smithi106 ceph-mon[29693]: purged_snaps scrub starts 2024-02-07T12:21:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:01 smithi106 ceph-mon[29693]: purged_snaps scrub ok 2024-02-07T12:21:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:01 smithi106 ceph-mon[29693]: osd.0 [v2:172.21.15.78:6802/1287860194,v1:172.21.15.78:6803/1287860194] boot 2024-02-07T12:21:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:01 smithi106 ceph-mon[29693]: osdmap e8: 1 total, 1 up, 1 in 2024-02-07T12:21:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:21:02.662 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi106:/dev/nvme4n1 2024-02-07T12:21:02.897 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.b/config 2024-02-07T12:21:02.992 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:02 smithi106 ceph-mon[29693]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:02.992 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:02 smithi106 ceph-mon[29693]: osdmap e9: 1 total, 1 up, 1 in 2024-02-07T12:21:03.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:02 smithi078 ceph-mon[25576]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:03.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:02 smithi078 ceph-mon[25576]: osdmap e9: 1 total, 1 up, 1 in 2024-02-07T12:21:03.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:02 smithi171 ceph-mon[30005]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:03.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:02 smithi171 ceph-mon[30005]: osdmap e9: 1 total, 1 up, 1 in 2024-02-07T12:21:04.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: Detected new or changed devices on smithi078 2024-02-07T12:21:04.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:21:04.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: Adjusting osd_memory_target on smithi078 to 17088M 2024-02-07T12:21:04.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:04.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:04.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:04 smithi106 ceph-mon[29693]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:04.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: Detected new or changed devices on smithi078 2024-02-07T12:21:04.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:21:04.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: Adjusting osd_memory_target on smithi078 to 17088M 2024-02-07T12:21:04.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:04.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:04.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:04 smithi078 ceph-mon[25576]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:04.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: Detected new or changed devices on smithi078 2024-02-07T12:21:04.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:21:04.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: Adjusting osd_memory_target on smithi078 to 17088M 2024-02-07T12:21:04.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:04.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:04.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:04.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:04 smithi171 ceph-mon[30005]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:05.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:05 smithi106 ceph-mon[29693]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:21:05.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:05 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T12:21:05.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:05 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T12:21:05.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:05 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:05.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:05 smithi078 ceph-mon[25576]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:21:05.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:05 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T12:21:05.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:05 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T12:21:05.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:05 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:05.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:05 smithi171 ceph-mon[30005]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:21:05.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:05 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T12:21:05.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:05 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T12:21:05.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:05 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:06.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:06 smithi106 ceph-mon[29693]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:06.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:06 smithi078 ceph-mon[25576]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:06.411 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:06 smithi171 ceph-mon[30005]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:07.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:07 smithi078 ceph-mon[25576]: from='client.? 172.21.15.106:0/4164319493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70389d5b-0c1a-4ca1-b03a-f25d3a682933"}]: dispatch 2024-02-07T12:21:07.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:07 smithi078 ceph-mon[25576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70389d5b-0c1a-4ca1-b03a-f25d3a682933"}]: dispatch 2024-02-07T12:21:07.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:07 smithi078 ceph-mon[25576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70389d5b-0c1a-4ca1-b03a-f25d3a682933"}]': finished 2024-02-07T12:21:07.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:07 smithi078 ceph-mon[25576]: osdmap e10: 2 total, 1 up, 2 in 2024-02-07T12:21:07.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:07 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:07.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:07 smithi171 ceph-mon[30005]: from='client.? 172.21.15.106:0/4164319493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70389d5b-0c1a-4ca1-b03a-f25d3a682933"}]: dispatch 2024-02-07T12:21:07.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:07 smithi171 ceph-mon[30005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70389d5b-0c1a-4ca1-b03a-f25d3a682933"}]: dispatch 2024-02-07T12:21:07.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:07 smithi171 ceph-mon[30005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70389d5b-0c1a-4ca1-b03a-f25d3a682933"}]': finished 2024-02-07T12:21:07.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:07 smithi171 ceph-mon[30005]: osdmap e10: 2 total, 1 up, 2 in 2024-02-07T12:21:07.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:07 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:07.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:07 smithi106 ceph-mon[29693]: from='client.? 172.21.15.106:0/4164319493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70389d5b-0c1a-4ca1-b03a-f25d3a682933"}]: dispatch 2024-02-07T12:21:07.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:07 smithi106 ceph-mon[29693]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70389d5b-0c1a-4ca1-b03a-f25d3a682933"}]: dispatch 2024-02-07T12:21:07.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:07 smithi106 ceph-mon[29693]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70389d5b-0c1a-4ca1-b03a-f25d3a682933"}]': finished 2024-02-07T12:21:07.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:07 smithi106 ceph-mon[29693]: osdmap e10: 2 total, 1 up, 2 in 2024-02-07T12:21:07.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:07 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:08.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:08 smithi078 ceph-mon[25576]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:08.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:08 smithi171 ceph-mon[30005]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:08.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:08 smithi106 ceph-mon[29693]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:09.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:09 smithi106 ceph-mon[29693]: from='client.? 172.21.15.106:0/1550884712' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T12:21:09.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:09 smithi078 ceph-mon[25576]: from='client.? 172.21.15.106:0/1550884712' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T12:21:09.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:09 smithi171 ceph-mon[30005]: from='client.? 172.21.15.106:0/1550884712' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T12:21:10.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:10 smithi078 ceph-mon[25576]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:10.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:10 smithi171 ceph-mon[30005]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:10.713 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:10 smithi106 ceph-mon[29693]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:12.996 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:12 smithi106 ceph-mon[29693]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:13.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:12 smithi078 ceph-mon[25576]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:13.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:12 smithi171 ceph-mon[30005]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:14.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:14 smithi106 ceph-mon[29693]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:14.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:14 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T12:21:14.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:14 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:15.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:14 smithi078 ceph-mon[25576]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:15.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:14 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T12:21:15.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:14 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:15.161 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:14 smithi171 ceph-mon[30005]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:15.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:14 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T12:21:15.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:14 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:16.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:15 smithi106 ceph-mon[29693]: Deploying daemon osd.1 on smithi106 2024-02-07T12:21:16.130 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:15 smithi078 ceph-mon[25576]: Deploying daemon osd.1 on smithi106 2024-02-07T12:21:16.161 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:15 smithi171 ceph-mon[30005]: Deploying daemon osd.1 on smithi106 2024-02-07T12:21:17.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:16 smithi078 ceph-mon[25576]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:17.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:16 smithi106 ceph-mon[29693]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:17.161 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:16 smithi171 ceph-mon[30005]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:18.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:18 smithi106 ceph-mon[29693]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:18.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:18 smithi078 ceph-mon[25576]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:18.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:18 smithi171 ceph-mon[30005]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:20.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:20 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:20.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:20 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:20.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:20 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:20.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:20 smithi106 ceph-mon[29693]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:20.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:20 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:20.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:20 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:20.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:20 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:20.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:20 smithi078 ceph-mon[25576]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:20.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:20 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:20.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:20 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:20.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:20 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:20.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:20 smithi171 ceph-mon[30005]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:22.709 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 1 on host 'smithi106' 2024-02-07T12:21:22.995 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:22.995 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:22.995 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:22.995 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:22.995 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:22.995 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:22.996 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='osd.1 [v2:172.21.15.106:6800/2287530274,v1:172.21.15.106:6801/2287530274]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T12:21:22.996 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T12:21:22.996 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:22.996 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:22.996 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:22 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:23.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:23.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:23.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:23.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='osd.1 [v2:172.21.15.106:6800/2287530274,v1:172.21.15.106:6801/2287530274]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T12:21:23.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T12:21:23.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:22 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:23.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:23.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:23.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:23.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='osd.1 [v2:172.21.15.106:6800/2287530274,v1:172.21.15.106:6801/2287530274]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T12:21:23.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T12:21:23.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:23.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:22 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:23.254 DEBUG:teuthology.orchestra.run.smithi106:osd.1> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@osd.1.service 2024-02-07T12:21:23.257 INFO:tasks.cephadm:Deploying osd.2 on smithi171 with /dev/nvme4n1... 2024-02-07T12:21:23.258 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-07T12:21:23.505 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:21:24.434 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:24 smithi106 ceph-mon[29693]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-07T12:21:24.434 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:24 smithi106 ceph-mon[29693]: from='osd.1 [v2:172.21.15.106:6800/2287530274,v1:172.21.15.106:6801/2287530274]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-02-07T12:21:24.434 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:24 smithi106 ceph-mon[29693]: osdmap e11: 2 total, 1 up, 2 in 2024-02-07T12:21:24.434 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:24 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:24.434 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:24 smithi106 ceph-mon[29693]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-02-07T12:21:24.434 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:24 smithi106 ceph-mon[29693]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:24.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:24 smithi078 ceph-mon[25576]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-07T12:21:24.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:24 smithi078 ceph-mon[25576]: from='osd.1 [v2:172.21.15.106:6800/2287530274,v1:172.21.15.106:6801/2287530274]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-02-07T12:21:24.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:24 smithi078 ceph-mon[25576]: osdmap e11: 2 total, 1 up, 2 in 2024-02-07T12:21:24.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:24 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:24.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:24 smithi078 ceph-mon[25576]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-02-07T12:21:24.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:24 smithi078 ceph-mon[25576]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:24.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:24 smithi171 ceph-mon[30005]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-07T12:21:24.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:24 smithi171 ceph-mon[30005]: from='osd.1 [v2:172.21.15.106:6800/2287530274,v1:172.21.15.106:6801/2287530274]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-02-07T12:21:24.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:24 smithi171 ceph-mon[30005]: osdmap e11: 2 total, 1 up, 2 in 2024-02-07T12:21:24.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:24 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:24.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:24 smithi171 ceph-mon[30005]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-02-07T12:21:24.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:24 smithi171 ceph-mon[30005]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:25.016 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:24 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1[34241]: 2024-02-07T12:21:24.834+0000 7ffb8dfd7700 -1 osd.1 0 waiting for initial osdmap 2024-02-07T12:21:25.016 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:24 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1[34241]: 2024-02-07T12:21:24.837+0000 7ffb855c8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-07T12:21:25.398 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/nvme4n1 2024-02-07T12:21:25.398 INFO:teuthology.orchestra.run.smithi171.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-07T12:21:25.398 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-07T12:21:25.399 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-02-07T12:21:25.399 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-02-07T12:21:25.399 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128277 s, 817 MB/s 2024-02-07T12:21:25.399 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-02-07T12:21:25.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:25 smithi078 ceph-mon[25576]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-02-07T12:21:25.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:25 smithi078 ceph-mon[25576]: osdmap e12: 2 total, 1 up, 2 in 2024-02-07T12:21:25.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:25.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:25 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:25.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:25 smithi078 ceph-mon[25576]: from='osd.1 ' entity='osd.1' 2024-02-07T12:21:25.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:25 smithi171 ceph-mon[30005]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-02-07T12:21:25.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:25 smithi171 ceph-mon[30005]: osdmap e12: 2 total, 1 up, 2 in 2024-02-07T12:21:25.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:25.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:25 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:25.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:25 smithi171 ceph-mon[30005]: from='osd.1 ' entity='osd.1' 2024-02-07T12:21:25.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:25 smithi106 ceph-mon[29693]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-02-07T12:21:25.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:25 smithi106 ceph-mon[29693]: osdmap e12: 2 total, 1 up, 2 in 2024-02-07T12:21:25.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:25 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:25.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:25 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:25.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:25 smithi106 ceph-mon[29693]: from='osd.1 ' entity='osd.1' 2024-02-07T12:21:25.970 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi171:/dev/nvme4n1 2024-02-07T12:21:26.202 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:21:26.564 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:26 smithi171 ceph-mon[30005]: purged_snaps scrub starts 2024-02-07T12:21:26.564 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:26 smithi171 ceph-mon[30005]: purged_snaps scrub ok 2024-02-07T12:21:26.564 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:26.564 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:26 smithi171 ceph-mon[30005]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:26.564 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:26 smithi171 ceph-mon[30005]: osd.1 [v2:172.21.15.106:6800/2287530274,v1:172.21.15.106:6801/2287530274] boot 2024-02-07T12:21:26.564 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:26 smithi171 ceph-mon[30005]: osdmap e13: 2 total, 2 up, 2 in 2024-02-07T12:21:26.564 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:26.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:26 smithi078 ceph-mon[25576]: purged_snaps scrub starts 2024-02-07T12:21:26.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:26 smithi078 ceph-mon[25576]: purged_snaps scrub ok 2024-02-07T12:21:26.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:26.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:26 smithi078 ceph-mon[25576]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:26.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:26 smithi078 ceph-mon[25576]: osd.1 [v2:172.21.15.106:6800/2287530274,v1:172.21.15.106:6801/2287530274] boot 2024-02-07T12:21:26.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:26 smithi078 ceph-mon[25576]: osdmap e13: 2 total, 2 up, 2 in 2024-02-07T12:21:26.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:26 smithi106 ceph-mon[29693]: purged_snaps scrub starts 2024-02-07T12:21:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:26 smithi106 ceph-mon[29693]: purged_snaps scrub ok 2024-02-07T12:21:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:26 smithi106 ceph-mon[29693]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T12:21:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:26 smithi106 ceph-mon[29693]: osd.1 [v2:172.21.15.106:6800/2287530274,v1:172.21.15.106:6801/2287530274] boot 2024-02-07T12:21:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:26 smithi106 ceph-mon[29693]: osdmap e13: 2 total, 2 up, 2 in 2024-02-07T12:21:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:21:27.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: Detected new or changed devices on smithi106 2024-02-07T12:21:27.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:21:27.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: Adjusting osd_memory_target on smithi106 to 17088M 2024-02-07T12:21:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:27 smithi078 ceph-mon[25576]: osdmap e14: 2 total, 2 up, 2 in 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: Detected new or changed devices on smithi106 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: Adjusting osd_memory_target on smithi106 to 17088M 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:27 smithi171 ceph-mon[30005]: osdmap e14: 2 total, 2 up, 2 in 2024-02-07T12:21:27.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: Detected new or changed devices on smithi106 2024-02-07T12:21:27.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:21:27.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: Adjusting osd_memory_target on smithi106 to 17088M 2024-02-07T12:21:27.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:27.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:27.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:27.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:27 smithi106 ceph-mon[29693]: osdmap e14: 2 total, 2 up, 2 in 2024-02-07T12:21:28.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:28 smithi171 ceph-mon[30005]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:28.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:28 smithi171 ceph-mon[30005]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:21:28.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:28 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T12:21:28.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:28 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T12:21:28.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:28 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:28.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:28 smithi106 ceph-mon[29693]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:28.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:28 smithi106 ceph-mon[29693]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:21:28.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:28 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T12:21:28.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:28 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T12:21:28.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:28 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:28.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:28 smithi078 ceph-mon[25576]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:28.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:28 smithi078 ceph-mon[25576]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:21:28.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:28 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T12:21:28.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:28 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T12:21:28.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:28 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:30 smithi078 ceph-mon[25576]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:30 smithi078 ceph-mon[25576]: from='client.? 172.21.15.171:0/2357858020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65"}]: dispatch 2024-02-07T12:21:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:30 smithi078 ceph-mon[25576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65"}]: dispatch 2024-02-07T12:21:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:30 smithi078 ceph-mon[25576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65"}]': finished 2024-02-07T12:21:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:30 smithi078 ceph-mon[25576]: osdmap e15: 3 total, 2 up, 3 in 2024-02-07T12:21:31.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:30 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:31.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:30 smithi171 ceph-mon[30005]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:31.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:30 smithi171 ceph-mon[30005]: from='client.? 172.21.15.171:0/2357858020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65"}]: dispatch 2024-02-07T12:21:31.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:30 smithi171 ceph-mon[30005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65"}]: dispatch 2024-02-07T12:21:31.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:30 smithi171 ceph-mon[30005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65"}]': finished 2024-02-07T12:21:31.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:30 smithi171 ceph-mon[30005]: osdmap e15: 3 total, 2 up, 3 in 2024-02-07T12:21:31.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:30 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:31.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:30 smithi106 ceph-mon[29693]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:31.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:30 smithi106 ceph-mon[29693]: from='client.? 172.21.15.171:0/2357858020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65"}]: dispatch 2024-02-07T12:21:31.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:30 smithi106 ceph-mon[29693]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65"}]: dispatch 2024-02-07T12:21:31.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:30 smithi106 ceph-mon[29693]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65"}]': finished 2024-02-07T12:21:31.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:30 smithi106 ceph-mon[29693]: osdmap e15: 3 total, 2 up, 3 in 2024-02-07T12:21:31.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:30 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:33.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:32 smithi078 ceph-mon[25576]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:33.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:32 smithi078 ceph-mon[25576]: from='client.? 172.21.15.171:0/3138300168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T12:21:33.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:32 smithi171 ceph-mon[30005]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:33.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:32 smithi171 ceph-mon[30005]: from='client.? 172.21.15.171:0/3138300168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T12:21:33.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:32 smithi106 ceph-mon[29693]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:33.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:32 smithi106 ceph-mon[29693]: from='client.? 172.21.15.171:0/3138300168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T12:21:35.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:34 smithi078 ceph-mon[25576]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:35.161 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:34 smithi171 ceph-mon[30005]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:35.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:34 smithi106 ceph-mon[29693]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:36.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:36 smithi171 ceph-mon[30005]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:37.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:36 smithi078 ceph-mon[25576]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:37.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:36 smithi106 ceph-mon[29693]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:38.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:38 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-07T12:21:38.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:38 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:38.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:38 smithi078 ceph-mon[25576]: Deploying daemon osd.2 on smithi171 2024-02-07T12:21:38.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:38 smithi078 ceph-mon[25576]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:38.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:38 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-07T12:21:38.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:38 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:38.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:38 smithi171 ceph-mon[30005]: Deploying daemon osd.2 on smithi171 2024-02-07T12:21:38.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:38 smithi171 ceph-mon[30005]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:38.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:38 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-07T12:21:38.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:38 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:38.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:38 smithi106 ceph-mon[29693]: Deploying daemon osd.2 on smithi171 2024-02-07T12:21:38.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:38 smithi106 ceph-mon[29693]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:41.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:40 smithi078 ceph-mon[25576]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:41.150 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:40 smithi171 ceph-mon[30005]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:41.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:40 smithi106 ceph-mon[29693]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:42 smithi171 ceph-mon[30005]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:43.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:42 smithi078 ceph-mon[25576]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:43.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:42 smithi106 ceph-mon[29693]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:44.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:44 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:44.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:44 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:44.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:44 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:44.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:44 smithi078 ceph-mon[25576]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:44.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:44 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:44.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:44 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:44.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:44 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:44.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:44 smithi171 ceph-mon[30005]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:44.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:44 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:44.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:44 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:44.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:44 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:44.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:44 smithi106 ceph-mon[29693]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:46.496 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 2 on host 'smithi171' 2024-02-07T12:21:47.066 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='osd.2 [v2:172.21.15.171:6800/4121485437,v1:172.21.15.171:6801/4121485437]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.067 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:46 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:47.084 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:47.084 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.084 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.084 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:47.084 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:47.084 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.085 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='osd.2 [v2:172.21.15.171:6800/4121485437,v1:172.21.15.171:6801/4121485437]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T12:21:47.085 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T12:21:47.085 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.085 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.085 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:46 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:47.125 DEBUG:teuthology.orchestra.run.smithi171:osd.2> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@osd.2.service 2024-02-07T12:21:47.127 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-02-07T12:21:47.127 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd stat -f json 2024-02-07T12:21:47.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='osd.2 [v2:172.21.15.171:6800/4121485437,v1:172.21.15.171:6801/4121485437]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:47.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:46 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:21:47.371 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:21:47.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:47 smithi171 ceph-mon[30005]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-07T12:21:47.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:47 smithi171 ceph-mon[30005]: osdmap e16: 3 total, 2 up, 3 in 2024-02-07T12:21:47.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:47 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:47.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:47 smithi171 ceph-mon[30005]: from='osd.2 [v2:172.21.15.171:6800/4121485437,v1:172.21.15.171:6801/4121485437]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-02-07T12:21:47.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:47 smithi171 ceph-mon[30005]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-02-07T12:21:48.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:47 smithi078 ceph-mon[25576]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-07T12:21:48.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:47 smithi078 ceph-mon[25576]: osdmap e16: 3 total, 2 up, 3 in 2024-02-07T12:21:48.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:47 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:48.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:47 smithi078 ceph-mon[25576]: from='osd.2 [v2:172.21.15.171:6800/4121485437,v1:172.21.15.171:6801/4121485437]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-02-07T12:21:48.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:47 smithi078 ceph-mon[25576]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-02-07T12:21:48.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:47 smithi106 ceph-mon[29693]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-07T12:21:48.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:47 smithi106 ceph-mon[29693]: osdmap e16: 3 total, 2 up, 3 in 2024-02-07T12:21:48.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:47 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:48.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:47 smithi106 ceph-mon[29693]: from='osd.2 [v2:172.21.15.171:6800/4121485437,v1:172.21.15.171:6801/4121485437]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-02-07T12:21:48.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:47 smithi106 ceph-mon[29693]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-02-07T12:21:48.559 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:48 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2[35269]: 2024-02-07T12:21:48.339+0000 7f627677a700 -1 osd.2 0 waiting for initial osdmap 2024-02-07T12:21:48.559 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:48 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2[35269]: 2024-02-07T12:21:48.342+0000 7f626e56c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-07T12:21:49.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:48 smithi106 ceph-mon[29693]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:49.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:48 smithi106 ceph-mon[29693]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-02-07T12:21:49.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:48 smithi106 ceph-mon[29693]: osdmap e17: 3 total, 2 up, 3 in 2024-02-07T12:21:49.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:48 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:49.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:48 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:49.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:48 smithi106 ceph-mon[29693]: from='osd.2 ' entity='osd.2' 2024-02-07T12:21:49.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:48 smithi078 ceph-mon[25576]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:49.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:48 smithi078 ceph-mon[25576]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-02-07T12:21:49.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:48 smithi078 ceph-mon[25576]: osdmap e17: 3 total, 2 up, 3 in 2024-02-07T12:21:49.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:49.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:48 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:49.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:48 smithi078 ceph-mon[25576]: from='osd.2 ' entity='osd.2' 2024-02-07T12:21:49.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:48 smithi171 ceph-mon[30005]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:49.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:48 smithi171 ceph-mon[30005]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-02-07T12:21:49.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:48 smithi171 ceph-mon[30005]: osdmap e17: 3 total, 2 up, 3 in 2024-02-07T12:21:49.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:48 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:49.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:48 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:49.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:48 smithi171 ceph-mon[30005]: from='osd.2 ' entity='osd.2' 2024-02-07T12:21:49.266 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:21:49.830 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"num_osds":3,"num_up_osds":2,"osd_up_since":1707308485,"num_in_osds":3,"osd_in_since":1707308490,"num_remapped_pgs":0} 2024-02-07T12:21:49.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:49 smithi171 ceph-mon[30005]: purged_snaps scrub starts 2024-02-07T12:21:49.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:49 smithi171 ceph-mon[30005]: purged_snaps scrub ok 2024-02-07T12:21:49.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:49 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:49.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:49 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1406680099' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T12:21:49.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:49 smithi171 ceph-mon[30005]: osd.2 [v2:172.21.15.171:6800/4121485437,v1:172.21.15.171:6801/4121485437] boot 2024-02-07T12:21:49.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:49 smithi171 ceph-mon[30005]: osdmap e18: 3 total, 3 up, 3 in 2024-02-07T12:21:49.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:49 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:50.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:49 smithi078 ceph-mon[25576]: purged_snaps scrub starts 2024-02-07T12:21:50.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:49 smithi078 ceph-mon[25576]: purged_snaps scrub ok 2024-02-07T12:21:50.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:49 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:50.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:49 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1406680099' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T12:21:50.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:49 smithi078 ceph-mon[25576]: osd.2 [v2:172.21.15.171:6800/4121485437,v1:172.21.15.171:6801/4121485437] boot 2024-02-07T12:21:50.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:49 smithi078 ceph-mon[25576]: osdmap e18: 3 total, 3 up, 3 in 2024-02-07T12:21:50.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:49 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:50.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:49 smithi106 ceph-mon[29693]: purged_snaps scrub starts 2024-02-07T12:21:50.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:49 smithi106 ceph-mon[29693]: purged_snaps scrub ok 2024-02-07T12:21:50.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:49 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:50.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:49 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1406680099' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T12:21:50.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:49 smithi106 ceph-mon[29693]: osd.2 [v2:172.21.15.171:6800/4121485437,v1:172.21.15.171:6801/4121485437] boot 2024-02-07T12:21:50.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:49 smithi106 ceph-mon[29693]: osdmap e18: 3 total, 3 up, 3 in 2024-02-07T12:21:50.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:49 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:21:50.832 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd stat -f json 2024-02-07T12:21:51.069 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:21:51.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:50 smithi078 ceph-mon[25576]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:51.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:50 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-07T12:21:51.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:50 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:50 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:50 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:21:51.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:50 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:50 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:51.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:50 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:51.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:50 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:50 smithi171 ceph-mon[30005]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:50 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-07T12:21:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:50 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:50 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:50 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:21:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:50 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:50 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:50 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:51.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:50 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:50 smithi106 ceph-mon[29693]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T12:21:51.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:50 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-07T12:21:51.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:50 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:50 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:50 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:21:51.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:50 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:51.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:50 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:21:51.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:50 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:21:51.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:50 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:52.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:51 smithi078 ceph-mon[25576]: Detected new or changed devices on smithi171 2024-02-07T12:21:52.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:51 smithi078 ceph-mon[25576]: Adjusting osd_memory_target on smithi171 to 17088M 2024-02-07T12:21:52.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:51 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-07T12:21:52.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:51 smithi078 ceph-mon[25576]: osdmap e19: 3 total, 3 up, 3 in 2024-02-07T12:21:52.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:51 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-07T12:21:52.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:51 smithi171 ceph-mon[30005]: Detected new or changed devices on smithi171 2024-02-07T12:21:52.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:51 smithi171 ceph-mon[30005]: Adjusting osd_memory_target on smithi171 to 17088M 2024-02-07T12:21:52.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:51 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-07T12:21:52.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:51 smithi171 ceph-mon[30005]: osdmap e19: 3 total, 3 up, 3 in 2024-02-07T12:21:52.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:51 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-07T12:21:52.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:51 smithi106 ceph-mon[29693]: Detected new or changed devices on smithi171 2024-02-07T12:21:52.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:51 smithi106 ceph-mon[29693]: Adjusting osd_memory_target on smithi171 to 17088M 2024-02-07T12:21:52.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:51 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-07T12:21:52.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:51 smithi106 ceph-mon[29693]: osdmap e19: 3 total, 3 up, 3 in 2024-02-07T12:21:52.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:51 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-07T12:21:52.461 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:52 smithi078 sudo[45479]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-07T12:21:52.461 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:52 smithi078 sudo[45479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:52.461 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:52 smithi078 sudo[45479]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:52.461 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:52 smithi078 sudo[45479]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:52.714 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:52 smithi078 sudo[45503]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-07T12:21:52.714 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:52 smithi078 sudo[45503]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:52.714 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:52 smithi078 sudo[45503]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:52.714 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:21:52 smithi078 sudo[45503]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:53.007 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:52 smithi106 sudo[39663]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-07T12:21:53.007 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:52 smithi106 sudo[39663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:53.007 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:52 smithi106 sudo[39663]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:53.007 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:52 smithi106 sudo[39663]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:53.007 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:52 smithi106 ceph-mon[29693]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:53.007 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:52 smithi106 ceph-mon[29693]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T12:21:53.007 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:52 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-07T12:21:53.007 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:52 smithi106 ceph-mon[29693]: osdmap e20: 3 total, 3 up, 3 in 2024-02-07T12:21:53.008 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:52 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:53.021 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:21:53.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:52 smithi078 ceph-mon[25576]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:53.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:52 smithi078 ceph-mon[25576]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T12:21:53.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:52 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-07T12:21:53.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:52 smithi078 ceph-mon[25576]: osdmap e20: 3 total, 3 up, 3 in 2024-02-07T12:21:53.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:52 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:53.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:52 smithi171 ceph-mon[30005]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:53.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:52 smithi171 ceph-mon[30005]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T12:21:53.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:52 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-07T12:21:53.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:52 smithi171 ceph-mon[30005]: osdmap e20: 3 total, 3 up, 3 in 2024-02-07T12:21:53.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:52 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:53.266 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:53 smithi106 sudo[39666]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-07T12:21:53.266 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:53 smithi106 sudo[39666]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:53.266 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:53 smithi106 sudo[39666]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:53.266 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:21:53 smithi106 sudo[39666]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:53.524 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:53 smithi171 sudo[40562]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-07T12:21:53.524 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:53 smithi171 sudo[40562]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:53.524 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:53 smithi171 sudo[40562]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:53.525 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:53 smithi171 sudo[40562]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:53.659 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":21,"num_osds":3,"num_up_osds":3,"osd_up_since":1707308509,"num_in_osds":3,"osd_in_since":1707308490,"num_remapped_pgs":0} 2024-02-07T12:21:53.660 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-07T12:21:53.899 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:21:53.912 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:53 smithi171 sudo[40565]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-07T12:21:53.912 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:53 smithi171 sudo[40565]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:53.912 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:53 smithi171 sudo[40565]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:53.912 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:21:53 smithi171 sudo[40565]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:54.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:53 smithi078 sudo[45591]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-07T12:21:54.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:53 smithi078 sudo[45591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:54.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:53 smithi078 sudo[45591]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:54.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:53 smithi078 sudo[45591]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:54.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 sudo[45651]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-07T12:21:54.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 sudo[45651]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:54.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 sudo[45651]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:54.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 sudo[45651]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:54.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 ceph-mon[25576]: osdmap e21: 3 total, 3 up, 3 in 2024-02-07T12:21:54.313 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3353443700' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T12:21:54.313 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:54.313 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 ceph-mon[25576]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-07T12:21:54.313 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:54.313 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:54 smithi078 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T12:21:54.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 ceph-mon[30005]: osdmap e21: 3 total, 3 up, 3 in 2024-02-07T12:21:54.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/3353443700' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T12:21:54.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:54.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 ceph-mon[30005]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-07T12:21:54.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:54.330 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 ceph-mon[30005]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T12:21:54.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 ceph-mon[29693]: osdmap e21: 3 total, 3 up, 3 in 2024-02-07T12:21:54.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/3353443700' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T12:21:54.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:54.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 ceph-mon[29693]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-07T12:21:54.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:54.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 ceph-mon[29693]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T12:21:54.581 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 sudo[40568]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-07T12:21:54.581 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 sudo[40568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:54.581 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 sudo[40568]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:54.581 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 sudo[40568]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:54.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 sudo[40571]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-07T12:21:54.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 sudo[40571]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:54.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 sudo[40571]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:54.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:54 smithi171 sudo[40571]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:55.094 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 sudo[39669]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-07T12:21:55.094 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 sudo[39669]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:55.094 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 sudo[39669]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:55.094 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:54 smithi106 sudo[39669]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:21:55.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:21:55.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T12:21:55.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: Cluster is now healthy 2024-02-07T12:21:55.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T12:21:55.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:55.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:21:55.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:21:55.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:55 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:21:55.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:55.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T12:21:55.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:21:55.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:21:55.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:21:55.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:55.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T12:21:55.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:21:55.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:21:55.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:21:55.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T12:21:55.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: Cluster is now healthy 2024-02-07T12:21:55.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T12:21:55.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:55.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:21:55.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:21:55.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:55 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:21:55.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 sudo[39672]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-07T12:21:55.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 sudo[39672]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T12:21:55.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 sudo[39672]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T12:21:55.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 sudo[39672]: pam_unix(sudo:session): session closed for user root 2024-02-07T12:21:55.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:55.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T12:21:55.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: Cluster is now healthy 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:21:55.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:55 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:21:55.780 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:21:55.780 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":21,"fsid":"dc35b7c8-c5b2-11ee-95b6-87774f69a715","created":"2024-02-07T12:18:27.439638+0000","modified":"2024-02-07T12:21:52.868986+0000","last_up_change":"2024-02-07T12:21:49.357504+0000","last_in_change":"2024-02-07T12:21:30.685927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-07T12:21:50.100646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e0658c29-df47-4181-b6a5-3aac7717f002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6803","nonce":1287860194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6805","nonce":1287860194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6809","nonce":1287860194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6807","nonce":1287860194}]},"public_addr":"172.21.15.78:6803/1287860194","cluster_addr":"172.21.15.78:6805/1287860194","heartbeat_back_addr":"172.21.15.78:6809/1287860194","heartbeat_front_addr":"172.21.15.78:6807/1287860194","state":["exists","up"]},{"osd":1,"uuid":"70389d5b-0c1a-4ca1-b03a-f25d3a682933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6801","nonce":2287530274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6803","nonce":2287530274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6807","nonce":2287530274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6805","nonce":2287530274}]},"public_addr":"172.21.15.106:6801/2287530274","cluster_addr":"172.21.15.106:6803/2287530274","heartbeat_back_addr":"172.21.15.106:6807/2287530274","heartbeat_front_addr":"172.21.15.106:6805/2287530274","state":["exists","up"]},{"osd":2,"uuid":"0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6801","nonce":4121485437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6803","nonce":4121485437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6807","nonce":4121485437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6805","nonce":4121485437}]},"public_addr":"172.21.15.171:6801/4121485437","cluster_addr":"172.21.15.171:6803/4121485437","heartbeat_back_addr":"172.21.15.171:6807/4121485437","heartbeat_front_addr":"172.21.15.171:6805/4121485437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T12:20:59.462789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T12:21:23.426598+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T12:21:47.186101+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3618557436":"2024-02-08T12:19:25.767855+0000","172.21.15.78:6800/3618557436":"2024-02-08T12:19:25.767855+0000","172.21.15.78:0/827542974":"2024-02-08T12:19:25.767855+0000","172.21.15.78:6800/510435962":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/3699092523":"2024-02-08T12:18:58.630853+0000","172.21.15.78:6801/510435962":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/321551342":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/2456231873":"2024-02-08T12:19:25.767855+0000","172.21.15.78:0/4213231788":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/1094866964":"2024-02-08T12:19:25.767855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-07T12:21:56.338 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:56 smithi078 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T12:21:56.338 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:56 smithi078 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T12:21:56.338 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:56 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:56.338 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:56 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2613311361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T12:21:56.338 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:56 smithi078 ceph-mon[25576]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:56.364 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-07T12:21:50.100646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-07T12:21:56.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-07T12:21:56.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:56 smithi171 ceph-mon[30005]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T12:21:56.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:56 smithi171 ceph-mon[30005]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T12:21:56.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:56 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:56.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:56 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/2613311361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T12:21:56.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:56 smithi171 ceph-mon[30005]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:56.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:56 smithi106 ceph-mon[29693]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T12:21:56.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:56 smithi106 ceph-mon[29693]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T12:21:56.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:56 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:21:56.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:56 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/2613311361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T12:21:56.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:56 smithi106 ceph-mon[29693]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:56.595 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:21:58.272 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-02-07T12:21:58.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:58 smithi078 ceph-mon[25576]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:58.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:21:58 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2222150659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-07T12:21:58.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:58 smithi171 ceph-mon[30005]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:58.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:21:58 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/2222150659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-07T12:21:58.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:58 smithi106 ceph-mon[29693]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:21:58.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:21:58 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/2222150659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-07T12:21:58.826 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi078 2024-02-07T12:21:58.827 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi078=foo.a' 2024-02-07T12:21:59.075 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:22:00.863 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled rgw.foo.a update... 2024-02-07T12:22:01.104 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:00 smithi171 ceph-mon[30005]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:01.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:00 smithi078 ceph-mon[25576]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:01.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:00 smithi106 ceph-mon[29693]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:01.373 DEBUG:teuthology.orchestra.run.smithi078:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@rgw.foo.a.service 2024-02-07T12:22:01.376 INFO:tasks.cephadm:Adding prometheus.a on smithi106 2024-02-07T12:22:01.376 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch apply prometheus '1;smithi106=a' 2024-02-07T12:22:01.610 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:22:01.983 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi078=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:01.983 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: Saving service rgw.foo.a spec with placement smithi078=foo.a;count:1 2024-02-07T12:22:01.983 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:01.983 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:01.984 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:01.984 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:22:01.984 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:01.984 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-07T12:22:01.984 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-07T12:22:01.984 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:01.984 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:01 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:02.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi078=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:02.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: Saving service rgw.foo.a spec with placement smithi078=foo.a;count:1 2024-02-07T12:22:02.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:02.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:02.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:02.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:22:02.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:02.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-07T12:22:02.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-07T12:22:02.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:02.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:01 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi078=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: Saving service rgw.foo.a spec with placement smithi078=foo.a;count:1 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:02.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:01 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:03.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:02 smithi078 ceph-mon[25576]: Deploying daemon rgw.foo.a on smithi078 2024-02-07T12:22:03.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:02 smithi078 ceph-mon[25576]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:03.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:02 smithi171 ceph-mon[30005]: Deploying daemon rgw.foo.a on smithi078 2024-02-07T12:22:03.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:02 smithi171 ceph-mon[30005]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:03.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:02 smithi106 ceph-mon[29693]: Deploying daemon rgw.foo.a on smithi078 2024-02-07T12:22:03.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:02 smithi106 ceph-mon[29693]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:03.419 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled prometheus update... 2024-02-07T12:22:03.631 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:22:03 smithi078 systemd[1]: Starting Ceph rgw.foo.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:22:03.926 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:22:03 smithi078 podman[46302]: 2024-02-07 12:22:03.564927925 +0000 UTC m=+0.017916443 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T12:22:03.927 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:22:03 smithi078 podman[46302]: 2024-02-07 12:22:03.68882089 +0000 UTC m=+0.141809391 container create 1c13524bc9740293b7cdb53a3577d0dc16e8955cd01b09eeae0a044dacdc410d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-07T12:22:03.927 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:22:03 smithi078 podman[46302]: 2024-02-07 12:22:03.924920413 +0000 UTC m=+0.377908929 container init 1c13524bc9740293b7cdb53a3577d0dc16e8955cd01b09eeae0a044dacdc410d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-07T12:22:03.998 DEBUG:teuthology.orchestra.run.smithi106:prometheus.a> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@prometheus.a.service 2024-02-07T12:22:04.001 INFO:tasks.cephadm:Adding node-exporter.a on smithi078 2024-02-07T12:22:04.001 INFO:tasks.cephadm:Adding node-exporter.b on smithi106 2024-02-07T12:22:04.001 INFO:tasks.cephadm:Adding node-exporter.c on smithi171 2024-02-07T12:22:04.001 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch apply node-exporter '3;smithi078=a;smithi106=b;smithi171=c' 2024-02-07T12:22:04.233 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:22:04.381 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:22:03 smithi078 podman[46302]: 2024-02-07 12:22:03.930568974 +0000 UTC m=+0.383557474 container start 1c13524bc9740293b7cdb53a3577d0dc16e8955cd01b09eeae0a044dacdc410d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-07T12:22:04.381 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:22:04 smithi078 bash[46302]: 1c13524bc9740293b7cdb53a3577d0dc16e8955cd01b09eeae0a044dacdc410d 2024-02-07T12:22:04.381 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:22:04 smithi078 systemd[1]: Started Ceph rgw.foo.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:22:04.624 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi106=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:04.624 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: Saving service prometheus spec with placement smithi106=a;count:1 2024-02-07T12:22:04.624 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.624 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:04.624 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.625 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.625 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.625 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.625 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.625 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:04 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi106=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: Saving service prometheus spec with placement smithi106=a;count:1 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:04 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi106=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: Saving service prometheus spec with placement smithi106=a;count:1 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:04.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:04 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:05.548 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:05 smithi171 ceph-mon[30005]: Saving service rgw.foo.a spec with placement smithi078=foo.a;count:1 2024-02-07T12:22:05.548 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:05 smithi171 ceph-mon[30005]: osdmap e22: 3 total, 3 up, 3 in 2024-02-07T12:22:05.548 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:05 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/2536490856' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-07T12:22:05.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:05 smithi078 ceph-mon[25576]: Saving service rgw.foo.a spec with placement smithi078=foo.a;count:1 2024-02-07T12:22:05.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:05 smithi078 ceph-mon[25576]: osdmap e22: 3 total, 3 up, 3 in 2024-02-07T12:22:05.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:05 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2536490856' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-07T12:22:05.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:05 smithi106 ceph-mon[29693]: Saving service rgw.foo.a spec with placement smithi078=foo.a;count:1 2024-02-07T12:22:05.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:05 smithi106 ceph-mon[29693]: osdmap e22: 3 total, 3 up, 3 in 2024-02-07T12:22:05.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:05 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/2536490856' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-07T12:22:06.191 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled node-exporter update... 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/2536490856' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: osdmap e23: 3 total, 3 up, 3 in 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: Checking dashboard <-> RGW credentials 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: pgmap v93: 33 pgs: 14 creating+peering, 1 active+clean, 18 unknown; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi078=a;smithi106=b;smithi171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: Saving service node-exporter spec with placement smithi078=a;smithi106=b;smithi171=c;count:3 2024-02-07T12:22:06.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:06 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/2536490856' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-07T12:22:06.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: osdmap e23: 3 total, 3 up, 3 in 2024-02-07T12:22:06.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:06.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:22:06.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: Checking dashboard <-> RGW credentials 2024-02-07T12:22:06.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: pgmap v93: 33 pgs: 14 creating+peering, 1 active+clean, 18 unknown; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:06.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi078=a;smithi106=b;smithi171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:06.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: Saving service node-exporter spec with placement smithi078=a;smithi106=b;smithi171=c;count:3 2024-02-07T12:22:06.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:06 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.831 DEBUG:teuthology.orchestra.run.smithi078:node-exporter.a> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.a.service 2024-02-07T12:22:06.834 DEBUG:teuthology.orchestra.run.smithi106:node-exporter.b> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.b.service 2024-02-07T12:22:06.837 DEBUG:teuthology.orchestra.run.smithi171:node-exporter.c> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.c.service 2024-02-07T12:22:06.839 INFO:tasks.cephadm:Adding alertmanager.a on smithi078 2024-02-07T12:22:06.839 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch apply alertmanager '1;smithi078=a' 2024-02-07T12:22:06.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2536490856' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-07T12:22:06.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: osdmap e23: 3 total, 3 up, 3 in 2024-02-07T12:22:06.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:06.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:22:06.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: Checking dashboard <-> RGW credentials 2024-02-07T12:22:06.859 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: pgmap v93: 33 pgs: 14 creating+peering, 1 active+clean, 18 unknown; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:06.859 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:06.859 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi078=a;smithi106=b;smithi171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:06.859 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: Saving service node-exporter spec with placement smithi078=a;smithi106=b;smithi171=c;count:3 2024-02-07T12:22:06.859 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:06 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:07.113 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:22:07.566 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:07 smithi171 ceph-mon[30005]: osdmap e24: 3 total, 3 up, 3 in 2024-02-07T12:22:07.567 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:07 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T12:22:07.567 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:07 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T12:22:07.567 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:07 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T12:22:07.567 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:07 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T12:22:07.567 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:07 smithi171 ceph-mon[30005]: osdmap e25: 3 total, 3 up, 3 in 2024-02-07T12:22:07.679 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:07 smithi106 ceph-mon[29693]: osdmap e24: 3 total, 3 up, 3 in 2024-02-07T12:22:07.679 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:07 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T12:22:07.679 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:07 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T12:22:07.679 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:07 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T12:22:07.680 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:07 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T12:22:07.680 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:07 smithi106 ceph-mon[29693]: osdmap e25: 3 total, 3 up, 3 in 2024-02-07T12:22:07.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:07 smithi078 ceph-mon[25576]: osdmap e24: 3 total, 3 up, 3 in 2024-02-07T12:22:07.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:07 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T12:22:07.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:07 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T12:22:07.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:07 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T12:22:07.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:07 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T12:22:07.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:07 smithi078 ceph-mon[25576]: osdmap e25: 3 total, 3 up, 3 in 2024-02-07T12:22:08.650 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:08 smithi171 ceph-mon[30005]: pgmap v96: 65 pgs: 19 active+clean, 26 creating+peering, 20 unknown; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-07T12:22:08.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:08 smithi106 ceph-mon[29693]: pgmap v96: 65 pgs: 19 active+clean, 26 creating+peering, 20 unknown; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-07T12:22:08.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:08 smithi078 ceph-mon[25576]: pgmap v96: 65 pgs: 19 active+clean, 26 creating+peering, 20 unknown; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-07T12:22:09.080 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled alertmanager update... 2024-02-07T12:22:09.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:09 smithi106 ceph-mon[29693]: osdmap e26: 3 total, 3 up, 3 in 2024-02-07T12:22:09.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:09 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T12:22:09.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:09 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T12:22:09.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:09 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:09.770 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:09 smithi171 ceph-mon[30005]: osdmap e26: 3 total, 3 up, 3 in 2024-02-07T12:22:09.770 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:09 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T12:22:09.770 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:09 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T12:22:09.770 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:09 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:09.797 DEBUG:teuthology.orchestra.run.smithi078:alertmanager.a> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@alertmanager.a.service 2024-02-07T12:22:09.800 INFO:tasks.cephadm:Adding grafana.a on smithi106 2024-02-07T12:22:09.801 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph orch apply grafana '1;smithi106=a' 2024-02-07T12:22:09.821 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:09 smithi078 ceph-mon[25576]: osdmap e26: 3 total, 3 up, 3 in 2024-02-07T12:22:09.822 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:09 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T12:22:09.822 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:09 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T12:22:09.822 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:09 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:10.035 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:22:10.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:10 smithi106 ceph-mon[29693]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi078=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:10.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:10 smithi106 ceph-mon[29693]: Saving service alertmanager spec with placement smithi078=a;count:1 2024-02-07T12:22:10.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:10 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T12:22:10.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:10 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T12:22:10.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:10 smithi106 ceph-mon[29693]: osdmap e27: 3 total, 3 up, 3 in 2024-02-07T12:22:10.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:10 smithi106 ceph-mon[29693]: pgmap v99: 97 pgs: 19 active+clean, 26 creating+peering, 52 unknown; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-07T12:22:10.868 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:10 smithi171 ceph-mon[30005]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi078=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:10.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:10 smithi171 ceph-mon[30005]: Saving service alertmanager spec with placement smithi078=a;count:1 2024-02-07T12:22:10.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:10 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T12:22:10.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:10 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T12:22:10.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:10 smithi171 ceph-mon[30005]: osdmap e27: 3 total, 3 up, 3 in 2024-02-07T12:22:10.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:10 smithi171 ceph-mon[30005]: pgmap v99: 97 pgs: 19 active+clean, 26 creating+peering, 52 unknown; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-07T12:22:10.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:10 smithi078 ceph-mon[25576]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi078=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:10.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:10 smithi078 ceph-mon[25576]: Saving service alertmanager spec with placement smithi078=a;count:1 2024-02-07T12:22:10.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:10 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T12:22:10.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:10 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T12:22:10.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:10 smithi078 ceph-mon[25576]: osdmap e27: 3 total, 3 up, 3 in 2024-02-07T12:22:10.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:10 smithi078 ceph-mon[25576]: pgmap v99: 97 pgs: 19 active+clean, 26 creating+peering, 52 unknown; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-07T12:22:11.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:11 smithi171 ceph-mon[30005]: osdmap e28: 3 total, 3 up, 3 in 2024-02-07T12:22:11.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:11 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T12:22:11.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:11 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T12:22:11.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:11 smithi106 ceph-mon[29693]: osdmap e28: 3 total, 3 up, 3 in 2024-02-07T12:22:11.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:11 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T12:22:11.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:11 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T12:22:11.821 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled grafana update... 2024-02-07T12:22:11.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:11 smithi078 ceph-mon[25576]: osdmap e28: 3 total, 3 up, 3 in 2024-02-07T12:22:11.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:11 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T12:22:11.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:11 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T12:22:12.395 DEBUG:teuthology.orchestra.run.smithi106:grafana.a> sudo journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@grafana.a.service 2024-02-07T12:22:12.398 INFO:tasks.cephadm:Setting up client nodes... 2024-02-07T12:22:12.399 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-07T12:22:12.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:12 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T12:22:12.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:12 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T12:22:12.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:12 smithi171 ceph-mon[30005]: osdmap e29: 3 total, 3 up, 3 in 2024-02-07T12:22:12.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:12 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T12:22:12.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:12 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T12:22:12.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:12 smithi171 ceph-mon[30005]: pgmap v102: 129 pgs: 107 active+clean, 13 creating+peering, 9 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-02-07T12:22:12.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:12 smithi171 ceph-mon[30005]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi106=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:12.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:12 smithi171 ceph-mon[30005]: Saving service grafana spec with placement smithi106=a;count:1 2024-02-07T12:22:12.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:12 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:12.697 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:12.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:12 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T12:22:12.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:12 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T12:22:12.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:12 smithi106 ceph-mon[29693]: osdmap e29: 3 total, 3 up, 3 in 2024-02-07T12:22:12.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:12 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T12:22:12.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:12 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T12:22:12.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:12 smithi106 ceph-mon[29693]: pgmap v102: 129 pgs: 107 active+clean, 13 creating+peering, 9 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-02-07T12:22:12.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:12 smithi106 ceph-mon[29693]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi106=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:12.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:12 smithi106 ceph-mon[29693]: Saving service grafana spec with placement smithi106=a;count:1 2024-02-07T12:22:12.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:12 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:12.823 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:12 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T12:22:12.823 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:12 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T12:22:12.823 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:12 smithi078 ceph-mon[25576]: osdmap e29: 3 total, 3 up, 3 in 2024-02-07T12:22:12.823 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:12 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T12:22:12.823 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:12 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T12:22:12.823 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:12 smithi078 ceph-mon[25576]: pgmap v102: 129 pgs: 107 active+clean, 13 creating+peering, 9 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-02-07T12:22:12.824 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:12 smithi078 ceph-mon[25576]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi106=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:22:12.824 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:12 smithi078 ceph-mon[25576]: Saving service grafana spec with placement smithi106=a;count:1 2024-02-07T12:22:12.824 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:12 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:13 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T12:22:13.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:13 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T12:22:13.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:13 smithi078 ceph-mon[25576]: osdmap e30: 3 total, 3 up, 3 in 2024-02-07T12:22:13.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:13 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:13 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:13 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:13 smithi078 ceph-mon[25576]: Deploying daemon prometheus.a on smithi106 2024-02-07T12:22:13.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:13 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T12:22:13.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:13 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T12:22:13.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:13 smithi106 ceph-mon[29693]: osdmap e30: 3 total, 3 up, 3 in 2024-02-07T12:22:13.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:13 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:13 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:13 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:13 smithi106 ceph-mon[29693]: Deploying daemon prometheus.a on smithi106 2024-02-07T12:22:13.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:13 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/4038004977' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T12:22:13.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:13 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1452111967' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T12:22:13.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:13 smithi171 ceph-mon[30005]: osdmap e30: 3 total, 3 up, 3 in 2024-02-07T12:22:13.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:13 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:13 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:13 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:13.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:13 smithi171 ceph-mon[30005]: Deploying daemon prometheus.a on smithi106 2024-02-07T12:22:14.495 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-02-07T12:22:14.496 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQD2dcNlLSNeHRAAApv1fGpgTitcKPTQBPUe4A== 2024-02-07T12:22:14.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:14 smithi106 ceph-mon[29693]: pgmap v104: 129 pgs: 107 active+clean, 13 creating+peering, 9 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-07T12:22:14.824 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:14 smithi078 ceph-mon[25576]: pgmap v104: 129 pgs: 107 active+clean, 13 creating+peering, 9 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-07T12:22:14.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:14 smithi171 ceph-mon[30005]: pgmap v104: 129 pgs: 107 active+clean, 13 creating+peering, 9 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-07T12:22:15.134 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:22:15.134 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-07T12:22:15.134 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-07T12:22:15.176 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-07T12:22:15.418 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.b/config 2024-02-07T12:22:15.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:15 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/2501896116' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:15.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:15 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/2501896116' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T12:22:15.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:15 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2501896116' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:15.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:15 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2501896116' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T12:22:15.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:15 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/2501896116' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:15.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:15 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/2501896116' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T12:22:16.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:16 smithi106 ceph-mon[29693]: pgmap v105: 129 pgs: 116 active+clean, 13 creating+peering; 579 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 26 KiB/s rd, 3.0 KiB/s wr, 58 op/s 2024-02-07T12:22:16.876 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:16 smithi171 ceph-mon[30005]: pgmap v105: 129 pgs: 116 active+clean, 13 creating+peering; 579 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 26 KiB/s rd, 3.0 KiB/s wr, 58 op/s 2024-02-07T12:22:16.880 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:16 smithi078 ceph-mon[25576]: pgmap v105: 129 pgs: 116 active+clean, 13 creating+peering; 579 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 26 KiB/s rd, 3.0 KiB/s wr, 58 op/s 2024-02-07T12:22:18.466 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:18 smithi106 ceph-mon[29693]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 63 KiB/s rd, 4.9 KiB/s wr, 153 op/s 2024-02-07T12:22:18.553 INFO:teuthology.orchestra.run.smithi106.stdout:[client.1] 2024-02-07T12:22:18.553 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQD6dcNlOsWgIBAA/4hWjbbtfSUO+Y8WvoSfeQ== 2024-02-07T12:22:18.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:18 smithi078 ceph-mon[25576]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 63 KiB/s rd, 4.9 KiB/s wr, 153 op/s 2024-02-07T12:22:18.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:18 smithi171 ceph-mon[30005]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 63 KiB/s rd, 4.9 KiB/s wr, 153 op/s 2024-02-07T12:22:19.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:19 smithi078 ceph-mon[25576]: from='client.? 172.21.15.106:0/3900233759' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:19.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:19 smithi078 ceph-mon[25576]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:19.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:19 smithi078 ceph-mon[25576]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T12:22:19.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:19 smithi171 ceph-mon[30005]: from='client.? 172.21.15.106:0/3900233759' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:19.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:19 smithi171 ceph-mon[30005]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:19.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:19 smithi171 ceph-mon[30005]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T12:22:19.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:19 smithi106 ceph-mon[29693]: from='client.? 172.21.15.106:0/3900233759' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:19.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:19 smithi106 ceph-mon[29693]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:19.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:19 smithi106 ceph-mon[29693]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T12:22:20.438 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:22:20.438 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-07T12:22:20.439 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-07T12:22:20.482 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-07T12:22:20.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:20 smithi078 ceph-mon[25576]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 55 KiB/s rd, 4.2 KiB/s wr, 134 op/s 2024-02-07T12:22:20.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:20 smithi171 ceph-mon[30005]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 55 KiB/s rd, 4.2 KiB/s wr, 134 op/s 2024-02-07T12:22:20.729 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T12:22:20.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:20 smithi106 ceph-mon[29693]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 55 KiB/s rd, 4.2 KiB/s wr, 134 op/s 2024-02-07T12:22:22.690 INFO:teuthology.orchestra.run.smithi171.stdout:[client.2] 2024-02-07T12:22:22.691 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQD+dcNlglTPKBAAzh07lHwzoGzZJBhanru6FA== 2024-02-07T12:22:23.010 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:22 smithi171 ceph-mon[30005]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-02-07T12:22:23.010 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:22 smithi171 ceph-mon[30005]: from='client.? 172.21.15.171:0/1686526550' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:23.010 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:22 smithi171 ceph-mon[30005]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:23.010 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:22 smithi171 ceph-mon[30005]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T12:22:23.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:22 smithi106 ceph-mon[29693]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-02-07T12:22:23.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:22 smithi106 ceph-mon[29693]: from='client.? 172.21.15.171:0/1686526550' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:23.082 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:22 smithi106 ceph-mon[29693]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:23.082 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:22 smithi106 ceph-mon[29693]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T12:22:23.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:22 smithi078 ceph-mon[25576]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-02-07T12:22:23.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:22 smithi078 ceph-mon[25576]: from='client.? 172.21.15.171:0/1686526550' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:23.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:22 smithi078 ceph-mon[25576]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T12:22:23.131 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:22 smithi078 ceph-mon[25576]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T12:22:23.287 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:22:23.287 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-02-07T12:22:23.287 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-07T12:22:23.329 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-07T12:22:23.390 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-07T12:22:23.390 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph mgr dump --format=json 2024-02-07T12:22:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:24.860 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:24 smithi106 systemd[1]: Starting Ceph prometheus.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:22:25.112 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:24 smithi106 ceph-mon[29693]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 3.0 KiB/s wr, 98 op/s 2024-02-07T12:22:25.113 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 podman[40587]: 2024-02-07 12:22:24.915269035 +0000 UTC m=+0.019578527 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-07T12:22:25.113 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 podman[40587]: 2024-02-07 12:22:25.013904137 +0000 UTC m=+0.118213617 container create 74e62edb26dca7cb20c3617b5bf3f3728255c8a8db360a4db0d0e6196a9ed452 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T12:22:25.130 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:24 smithi078 ceph-mon[25576]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 3.0 KiB/s wr, 98 op/s 2024-02-07T12:22:25.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:24 smithi171 ceph-mon[30005]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 3.0 KiB/s wr, 98 op/s 2024-02-07T12:22:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:22:25.516 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 podman[40587]: 2024-02-07 12:22:25.207692866 +0000 UTC m=+0.312002355 container init 74e62edb26dca7cb20c3617b5bf3f3728255c8a8db360a4db0d0e6196a9ed452 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T12:22:25.516 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 podman[40587]: 2024-02-07 12:22:25.213192473 +0000 UTC m=+0.317501953 container start 74e62edb26dca7cb20c3617b5bf3f3728255c8a8db360a4db0d0e6196a9ed452 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T12:22:25.516 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.236Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-07T12:22:25.516 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.236Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-07T12:22:25.516 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.236Z caller=main.go:570 level=info host_details="(Linux 5.14.0-418.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Feb 5 23:17:48 UTC 2024 x86_64 smithi106 (none))" 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.236Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.236Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.238Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.238Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.239Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.239Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.240Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.240Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.601µs 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.240Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.241Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.241Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.182µs wal_replay_duration=559.906µs wbl_replay_duration=146ns total_replay_duration=599.282µs 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.242Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.242Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-07T12:22:25.517 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.242Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-07T12:22:25.518 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.260Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=17.508372ms db_storage=709ns remote_storage=1.072µs web_handler=349ns query_engine=458ns scrape=398.133µs scrape_sd=76.347µs notify=946ns notify_sd=2.048µs rules=16.851509ms tracing=5.045µs 2024-02-07T12:22:25.518 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.260Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-07T12:22:25.518 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:22:25.260Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-07T12:22:25.518 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 bash[40587]: 74e62edb26dca7cb20c3617b5bf3f3728255c8a8db360a4db0d0e6196a9ed452 2024-02-07T12:22:25.518 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:22:25 smithi106 systemd[1]: Started Ceph prometheus.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:22:26.227 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4036548438},{"type":"v1","addr":"172.21.15.78:6801","nonce":4036548438}]},"active_addr":"172.21.15.78:6801/4036548438","active_change":"2024-02-07T12:19:25.767963+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"x","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.78:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":60896422}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":55168318}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":3664926145}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":3161424507}]}]} 2024-02-07T12:22:26.232 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-07T12:22:26.232 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-07T12:22:26.233 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-07T12:22:26.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.446 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.446 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.446 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-07T12:22:26.446 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1581515951' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-07T12:22:26.446 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-mon[25576]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.8 KiB/s wr, 92 op/s 2024-02-07T12:22:26.446 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:26.662 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: ignoring --setuser ceph since I am not root 2024-02-07T12:22:26.662 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: ignoring --setgroup ceph since I am not root 2024-02-07T12:22:26.662 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-mgr[32102]: -- 172.21.15.171:0/1095220056 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56536d121520 con 0x56536d0f6800 2024-02-07T12:22:26.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-07T12:22:26.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1581515951' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-07T12:22:26.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-mon[30005]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.8 KiB/s wr, 92 op/s 2024-02-07T12:22:26.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.739 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ignoring --setuser ceph since I am not root 2024-02-07T12:22:26.739 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ignoring --setgroup ceph since I am not root 2024-02-07T12:22:26.765 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: ignoring --setuser ceph since I am not root 2024-02-07T12:22:26.766 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: ignoring --setgroup ceph since I am not root 2024-02-07T12:22:26.766 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-mgr[31175]: -- 172.21.15.106:0/3277217225 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562c18e07080 con 0x562c18dd1000 2024-02-07T12:22:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-07T12:22:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1581515951' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-07T12:22:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-mon[29693]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.8 KiB/s wr, 92 op/s 2024-02-07T12:22:26.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' 2024-02-07T12:22:27.070 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:26 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:26.974+0000 7f33c3c40200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T12:22:27.070 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:27 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:27.067+0000 7f33c3c40200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T12:22:27.265 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:26 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:26.936+0000 7f9896986200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T12:22:27.266 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:27 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:27.029+0000 7f9896986200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T12:22:27.377 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:26 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:26.948+0000 7f915cb77200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T12:22:27.378 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:27 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:27.042+0000 7f915cb77200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T12:22:27.609 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:27 smithi078 ceph-mon[25576]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-07T12:22:27.609 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:27 smithi078 ceph-mon[25576]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-07T12:22:27.660 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:27 smithi106 ceph-mon[29693]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-07T12:22:27.660 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:27 smithi106 ceph-mon[29693]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-07T12:22:27.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:27 smithi171 ceph-mon[30005]: from='mgr.14150 172.21.15.78:0/32337666' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-07T12:22:27.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:27 smithi171 ceph-mon[30005]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-07T12:22:27.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:27 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:27.721+0000 7f33c3c40200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T12:22:27.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:27 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:27.813+0000 7f33c3c40200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T12:22:27.982 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:27 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:27.657+0000 7f9896986200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T12:22:27.982 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:27 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:27.748+0000 7f9896986200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T12:22:27.982 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:27 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:27.887+0000 7f9896986200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T12:22:28.004 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:27 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:27.675+0000 7f915cb77200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T12:22:28.005 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:27 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:27.767+0000 7f915cb77200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T12:22:28.005 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:27 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:27.907+0000 7f915cb77200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T12:22:28.005 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:28 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:28.003+0000 7f915cb77200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T12:22:28.265 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:27 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:27.979+0000 7f9896986200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T12:22:28.266 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:28 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:28.062+0000 7f9896986200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T12:22:28.350 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:28 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:28.086+0000 7f915cb77200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T12:22:28.350 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:28 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:28.348+0000 7f915cb77200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T12:22:28.375 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:22:28.375 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":30,"fsid":"dc35b7c8-c5b2-11ee-95b6-87774f69a715","created":"2024-02-07T12:18:27.439638+0000","modified":"2024-02-07T12:22:12.472259+0000","last_up_change":"2024-02-07T12:21:49.357504+0000","last_in_change":"2024-02-07T12:21:30.685927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-07T12:21:50.100646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-07T12:22:04.205664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-07T12:22:05.750105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-07T12:22:07.682654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-07T12:22:09.753096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e0658c29-df47-4181-b6a5-3aac7717f002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6803","nonce":1287860194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6805","nonce":1287860194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6809","nonce":1287860194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6807","nonce":1287860194}]},"public_addr":"172.21.15.78:6803/1287860194","cluster_addr":"172.21.15.78:6805/1287860194","heartbeat_back_addr":"172.21.15.78:6809/1287860194","heartbeat_front_addr":"172.21.15.78:6807/1287860194","state":["exists","up"]},{"osd":1,"uuid":"70389d5b-0c1a-4ca1-b03a-f25d3a682933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6801","nonce":2287530274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6803","nonce":2287530274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6807","nonce":2287530274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6805","nonce":2287530274}]},"public_addr":"172.21.15.106:6801/2287530274","cluster_addr":"172.21.15.106:6803/2287530274","heartbeat_back_addr":"172.21.15.106:6807/2287530274","heartbeat_front_addr":"172.21.15.106:6805/2287530274","state":["exists","up"]},{"osd":2,"uuid":"0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6801","nonce":4121485437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6803","nonce":4121485437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6807","nonce":4121485437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6805","nonce":4121485437}]},"public_addr":"172.21.15.171:6801/4121485437","cluster_addr":"172.21.15.171:6803/4121485437","heartbeat_back_addr":"172.21.15.171:6807/4121485437","heartbeat_front_addr":"172.21.15.171:6805/4121485437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T12:20:59.462789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T12:21:23.426598+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T12:21:47.186101+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3618557436":"2024-02-08T12:19:25.767855+0000","172.21.15.78:6800/3618557436":"2024-02-08T12:19:25.767855+0000","172.21.15.78:0/827542974":"2024-02-08T12:19:25.767855+0000","172.21.15.78:6800/510435962":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/3699092523":"2024-02-08T12:18:58.630853+0000","172.21.15.78:6801/510435962":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/321551342":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/2456231873":"2024-02-08T12:19:25.767855+0000","172.21.15.78:0/4213231788":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/1094866964":"2024-02-08T12:19:25.767855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-07T12:22:28.375 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:27 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:27.957+0000 7f33c3c40200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T12:22:28.375 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:28 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:28.054+0000 7f33c3c40200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T12:22:28.375 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:28 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:28.141+0000 7f33c3c40200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T12:22:28.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:28 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/2513911917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T12:22:28.610 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:28 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:28.322+0000 7f9896986200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T12:22:28.610 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:28 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:28.430+0000 7f9896986200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T12:22:28.610 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:28 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:28.514+0000 7f9896986200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T12:22:28.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:28 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2513911917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T12:22:28.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:28 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:28.414+0000 7f33c3c40200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T12:22:28.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:28 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:28.525+0000 7f33c3c40200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T12:22:28.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:28 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:28.614+0000 7f33c3c40200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T12:22:28.641 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:28 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/2513911917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T12:22:28.641 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:28 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:28.459+0000 7f915cb77200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T12:22:28.642 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:28 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:28.544+0000 7f915cb77200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T12:22:28.642 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:28 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:28.639+0000 7f915cb77200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T12:22:28.912 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:28 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:28.816+0000 7f915cb77200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T12:22:29.015 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:28 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:28.607+0000 7f9896986200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T12:22:29.015 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:28 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:28.781+0000 7f9896986200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T12:22:29.074 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:28 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:28.710+0000 7f33c3c40200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T12:22:29.074 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:28 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:28.885+0000 7f33c3c40200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T12:22:29.101 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-07T12:22:29.101 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-07T12:22:29.372 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:30.015 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:29 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:29.732+0000 7f9896986200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T12:22:30.016 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:29 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:29.860+0000 7f9896986200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T12:22:30.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:29 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:29.888+0000 7f33c3c40200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T12:22:30.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:30 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:30.019+0000 7f33c3c40200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T12:22:30.162 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:29 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:29.779+0000 7f915cb77200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T12:22:30.162 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:29 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:29.909+0000 7f915cb77200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T12:22:30.456 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:30 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:30.359+0000 7f33c3c40200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T12:22:30.456 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:30 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:30.454+0000 7f33c3c40200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T12:22:30.477 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:30 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:30.193+0000 7f9896986200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T12:22:30.477 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:30 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:30.283+0000 7f9896986200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T12:22:30.531 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:30 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:30.244+0000 7f915cb77200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T12:22:30.531 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:30 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:30.333+0000 7f915cb77200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T12:22:30.531 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:30 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:30.529+0000 7f915cb77200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:22:30.716 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:30 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:30.654+0000 7f33c3c40200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:22:30.765 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:30 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:30.475+0000 7f9896986200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:22:30.766 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:30 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:30.658+0000 7f9896986200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T12:22:30.912 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:30 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:30.714+0000 7f915cb77200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T12:22:31.131 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:30 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:30.854+0000 7f33c3c40200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T12:22:31.159 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:22:31.159 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":30,"fsid":"dc35b7c8-c5b2-11ee-95b6-87774f69a715","created":"2024-02-07T12:18:27.439638+0000","modified":"2024-02-07T12:22:12.472259+0000","last_up_change":"2024-02-07T12:21:49.357504+0000","last_in_change":"2024-02-07T12:21:30.685927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-07T12:21:50.100646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-07T12:22:04.205664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-07T12:22:05.750105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-07T12:22:07.682654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-07T12:22:09.753096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e0658c29-df47-4181-b6a5-3aac7717f002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6803","nonce":1287860194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6805","nonce":1287860194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6809","nonce":1287860194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1287860194},{"type":"v1","addr":"172.21.15.78:6807","nonce":1287860194}]},"public_addr":"172.21.15.78:6803/1287860194","cluster_addr":"172.21.15.78:6805/1287860194","heartbeat_back_addr":"172.21.15.78:6809/1287860194","heartbeat_front_addr":"172.21.15.78:6807/1287860194","state":["exists","up"]},{"osd":1,"uuid":"70389d5b-0c1a-4ca1-b03a-f25d3a682933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6801","nonce":2287530274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6803","nonce":2287530274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6807","nonce":2287530274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2287530274},{"type":"v1","addr":"172.21.15.106:6805","nonce":2287530274}]},"public_addr":"172.21.15.106:6801/2287530274","cluster_addr":"172.21.15.106:6803/2287530274","heartbeat_back_addr":"172.21.15.106:6807/2287530274","heartbeat_front_addr":"172.21.15.106:6805/2287530274","state":["exists","up"]},{"osd":2,"uuid":"0fbb2d4d-9ec9-4ba6-a62f-e145fb3f3c65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6801","nonce":4121485437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6803","nonce":4121485437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6807","nonce":4121485437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":4121485437},{"type":"v1","addr":"172.21.15.171:6805","nonce":4121485437}]},"public_addr":"172.21.15.171:6801/4121485437","cluster_addr":"172.21.15.171:6803/4121485437","heartbeat_back_addr":"172.21.15.171:6807/4121485437","heartbeat_front_addr":"172.21.15.171:6805/4121485437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T12:20:59.462789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T12:21:23.426598+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T12:21:47.186101+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3618557436":"2024-02-08T12:19:25.767855+0000","172.21.15.78:6800/3618557436":"2024-02-08T12:19:25.767855+0000","172.21.15.78:0/827542974":"2024-02-08T12:19:25.767855+0000","172.21.15.78:6800/510435962":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/3699092523":"2024-02-08T12:18:58.630853+0000","172.21.15.78:6801/510435962":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/321551342":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/2456231873":"2024-02-08T12:19:25.767855+0000","172.21.15.78:0/4213231788":"2024-02-08T12:18:58.630853+0000","172.21.15.78:0/1094866964":"2024-02-08T12:19:25.767855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-07T12:22:31.196 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:30 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:30.938+0000 7f9896986200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:22:31.263 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:31 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/3788152552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T12:22:31.264 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:31 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:31.000+0000 7f915cb77200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:22:31.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:31 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3788152552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T12:22:31.410 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:31 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:31.147+0000 7f33c3c40200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T12:22:31.410 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:31 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:31.407+0000 7f33c3c40200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T12:22:31.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:31 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/3788152552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T12:22:31.516 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:31 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:31.194+0000 7f9896986200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T12:22:31.516 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:31 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:31.429+0000 7f9896986200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T12:22:31.661 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:31 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:31.261+0000 7f915cb77200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T12:22:31.662 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:31 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:31.498+0000 7f915cb77200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T12:22:31.802 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:31 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:31.652+0000 7f33c3c40200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T12:22:31.831 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-07T12:22:31.832 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-07T12:22:31.832 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-07T12:22:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:32.113 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:32.515 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:32 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:32.219+0000 7f9896986200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T12:22:32.661 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:32 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:32.295+0000 7f915cb77200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T12:22:32.800 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:32 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:32.521+0000 7f33c3c40200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T12:22:33.104 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:32 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:32.786+0000 7f9896986200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T12:22:33.104 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:32 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:32.884+0000 7f9896986200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T12:22:33.161 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:32 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:32.868+0000 7f915cb77200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T12:22:33.162 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:32 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:32.966+0000 7f915cb77200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T12:22:33.371 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:33.104+0000 7f33c3c40200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T12:22:33.372 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:33.205+0000 7f33c3c40200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T12:22:33.372 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:33.101+0000 7f9896986200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T12:22:33.373 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: [07/Feb/2024:12:22:33] ENGINE Bus STARTING 2024-02-07T12:22:33.373 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: CherryPy Checker: 2024-02-07T12:22:33.373 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: The Application mounted at '' has an empty config. 2024-02-07T12:22:33.373 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: 2024-02-07T12:22:33.373 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: [07/Feb/2024:12:22:33] ENGINE Serving on http://:::9283 2024-02-07T12:22:33.373 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x[31152]: [07/Feb/2024:12:22:33] ENGINE Bus STARTED 2024-02-07T12:22:33.612 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:33.185+0000 7f915cb77200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T12:22:33.612 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: [07/Feb/2024:12:22:33] ENGINE Bus STARTING 2024-02-07T12:22:33.612 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: CherryPy Checker: 2024-02-07T12:22:33.612 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: The Application mounted at '' has an empty config. 2024-02-07T12:22:33.612 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: [07/Feb/2024:12:22:33] ENGINE Serving on http://:::9283 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z[32077]: [07/Feb/2024:12:22:33] ENGINE Bus STARTED 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: Standby manager daemon x restarted 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: Standby manager daemon x started 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: Standby manager daemon z restarted 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: Standby manager daemon z started 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T12:22:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:22:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T12:22:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:33 smithi171 ceph-mon[30005]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:22:33.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:33.430+0000 7f33c3c40200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T12:22:33.631 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:22:33] ENGINE Bus STARTING 2024-02-07T12:22:33.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: Standby manager daemon x restarted 2024-02-07T12:22:33.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: Standby manager daemon x started 2024-02-07T12:22:33.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T12:22:33.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:22:33.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T12:22:33.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:22:33.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: Standby manager daemon z restarted 2024-02-07T12:22:33.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: Standby manager daemon z started 2024-02-07T12:22:33.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T12:22:33.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:22:33.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T12:22:33.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-mon[25576]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: Standby manager daemon x restarted 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: Standby manager daemon x started 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.106:0/1954211079' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: Standby manager daemon z restarted 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: Standby manager daemon z started 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T12:22:33.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T12:22:33.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T12:22:33.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:33 smithi106 ceph-mon[29693]: from='mgr.? 172.21.15.171:0/289437733' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T12:22:34.106 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: CherryPy Checker: 2024-02-07T12:22:34.106 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: The Application mounted at '' has an empty config. 2024-02-07T12:22:34.107 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:22:34.107 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:22:33] ENGINE Serving on http://:::9283 2024-02-07T12:22:34.107 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:33 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:22:33] ENGINE Bus STARTED 2024-02-07T12:22:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: mgrmap e18: y(active, since 3m), standbys: z, x 2024-02-07T12:22:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: Active manager daemon y restarted 2024-02-07T12:22:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: Activating manager daemon y 2024-02-07T12:22:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: osdmap e31: 3 total, 3 up, 3 in 2024-02-07T12:22:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: mgrmap e19: y(active, starting, since 0.0103384s), standbys: z, x 2024-02-07T12:22:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:22:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:22:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: Manager daemon y is now available 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:22:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T12:22:34.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:34 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: mgrmap e18: y(active, since 3m), standbys: z, x 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: Active manager daemon y restarted 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: Activating manager daemon y 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: osdmap e31: 3 total, 3 up, 3 in 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: mgrmap e19: y(active, starting, since 0.0103384s), standbys: z, x 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T12:22:34.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: Manager daemon y is now available 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T12:22:34.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:34 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T12:22:34.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: mgrmap e18: y(active, since 3m), standbys: z, x 2024-02-07T12:22:34.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: Active manager daemon y restarted 2024-02-07T12:22:34.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: Activating manager daemon y 2024-02-07T12:22:34.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: osdmap e31: 3 total, 3 up, 3 in 2024-02-07T12:22:34.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: mgrmap e19: y(active, starting, since 0.0103384s), standbys: z, x 2024-02-07T12:22:34.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: Manager daemon y is now available 2024-02-07T12:22:34.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:34.768 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T12:22:34.768 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:34.768 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:22:34.768 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T12:22:34.768 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:34 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T12:22:35.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:35 smithi171 ceph-mon[30005]: [07/Feb/2024:12:22:34] ENGINE Bus STARTING 2024-02-07T12:22:35.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:35 smithi171 ceph-mon[30005]: [07/Feb/2024:12:22:34] ENGINE Serving on https://172.21.15.78:7150 2024-02-07T12:22:35.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:35 smithi171 ceph-mon[30005]: mgrmap e20: y(active, since 1.01688s), standbys: z, x 2024-02-07T12:22:35.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:35 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:35.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:35 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:35.489 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:35 smithi078 ceph-mon[25576]: [07/Feb/2024:12:22:34] ENGINE Bus STARTING 2024-02-07T12:22:35.489 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:35 smithi078 ceph-mon[25576]: [07/Feb/2024:12:22:34] ENGINE Serving on https://172.21.15.78:7150 2024-02-07T12:22:35.489 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:35 smithi078 ceph-mon[25576]: mgrmap e20: y(active, since 1.01688s), standbys: z, x 2024-02-07T12:22:35.489 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:35 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:35.489 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:35 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:35.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:35 smithi106 ceph-mon[29693]: [07/Feb/2024:12:22:34] ENGINE Bus STARTING 2024-02-07T12:22:35.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:35 smithi106 ceph-mon[29693]: [07/Feb/2024:12:22:34] ENGINE Serving on https://172.21.15.78:7150 2024-02-07T12:22:35.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:35 smithi106 ceph-mon[29693]: mgrmap e20: y(active, since 1.01688s), standbys: z, x 2024-02-07T12:22:35.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:35 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:35.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:35 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:36.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:36 smithi078 ceph-mon[25576]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:36.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:36 smithi078 ceph-mon[25576]: [07/Feb/2024:12:22:34] ENGINE Serving on http://172.21.15.78:8765 2024-02-07T12:22:36.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:36 smithi078 ceph-mon[25576]: [07/Feb/2024:12:22:34] ENGINE Bus STARTED 2024-02-07T12:22:36.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:36 smithi171 ceph-mon[30005]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:36.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:36 smithi171 ceph-mon[30005]: [07/Feb/2024:12:22:34] ENGINE Serving on http://172.21.15.78:8765 2024-02-07T12:22:36.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:36 smithi171 ceph-mon[30005]: [07/Feb/2024:12:22:34] ENGINE Bus STARTED 2024-02-07T12:22:36.758 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:36 smithi106 ceph-mon[29693]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:36.758 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:36 smithi106 ceph-mon[29693]: [07/Feb/2024:12:22:34] ENGINE Serving on http://172.21.15.78:8765 2024-02-07T12:22:36.758 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:36 smithi106 ceph-mon[29693]: [07/Feb/2024:12:22:34] ENGINE Bus STARTED 2024-02-07T12:22:36.951 INFO:teuthology.orchestra.run.smithi078.stdout:34359738389 2024-02-07T12:22:36.952 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-07T12:22:37.586 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:37 smithi078 ceph-mon[25576]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:37.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:37 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:37 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:37 smithi078 ceph-mon[25576]: mgrmap e21: y(active, since 3s), standbys: z, x 2024-02-07T12:22:37.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:37 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:37 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.612 INFO:teuthology.orchestra.run.smithi078.stdout:77309411340 2024-02-07T12:22:37.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-07T12:22:37.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:37 smithi171 ceph-mon[30005]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:37.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:37 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:37 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:37 smithi171 ceph-mon[30005]: mgrmap e21: y(active, since 3s), standbys: z, x 2024-02-07T12:22:37.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:37 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:37 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:37 smithi106 ceph-mon[29693]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:37.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:37 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:37 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:37 smithi106 ceph-mon[29693]: mgrmap e21: y(active, since 3s), standbys: z, x 2024-02-07T12:22:37.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:37 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:37.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:37 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:38.068 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:38.528 INFO:teuthology.orchestra.run.smithi078.stdout:55834574865 2024-02-07T12:22:38.528 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-07T12:22:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:39.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:39 smithi078 ceph-mon[25576]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:39.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:39 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:39.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:39 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:39.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:39 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:22:39.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:39 smithi078 ceph-mon[25576]: Adjusting osd_memory_target on smithi106 to 16064M 2024-02-07T12:22:39.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:39 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:39.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:39 smithi078 ceph-mon[25576]: mgrmap e22: y(active, since 4s), standbys: z, x 2024-02-07T12:22:39.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:39 smithi106 ceph-mon[29693]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:39.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:39 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:39.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:39 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:39.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:39 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:22:39.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:39 smithi106 ceph-mon[29693]: Adjusting osd_memory_target on smithi106 to 16064M 2024-02-07T12:22:39.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:39 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:39.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:39 smithi106 ceph-mon[29693]: mgrmap e22: y(active, since 4s), standbys: z, x 2024-02-07T12:22:39.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:39 smithi171 ceph-mon[30005]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:39.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:39 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:39.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:39 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:39.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:39 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:22:39.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:39 smithi171 ceph-mon[30005]: Adjusting osd_memory_target on smithi106 to 16064M 2024-02-07T12:22:39.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:39 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:39.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:39 smithi171 ceph-mon[30005]: mgrmap e22: y(active, since 4s), standbys: z, x 2024-02-07T12:22:40.463 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:40.664 INFO:teuthology.orchestra.run.smithi078.stdout:77309411340 2024-02-07T12:22:41.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:41 smithi106 ceph-mon[29693]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-07T12:22:41.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:41 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/1551202509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-07T12:22:41.525 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:41 smithi078 ceph-mon[25576]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-07T12:22:41.525 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:41 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/1551202509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-07T12:22:41.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:41 smithi171 ceph-mon[30005]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-07T12:22:41.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:41 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/1551202509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-07T12:22:41.809 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411340 got 77309411340 for osd.2 2024-02-07T12:22:41.809 DEBUG:teuthology.parallel:result is None 2024-02-07T12:22:42.594 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:42 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ::ffff:172.21.15.106 - - [07/Feb/2024:12:22:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T12:22:43.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:43 smithi078 ceph-mon[25576]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-07T12:22:43.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:43 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:43.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:43 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:43.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:43 smithi171 ceph-mon[30005]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-07T12:22:43.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:43 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:43.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:43 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:43.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:43 smithi106 ceph-mon[29693]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-07T12:22:43.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:43 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:43.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:43 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:43.790 INFO:teuthology.orchestra.run.smithi078.stdout:55834574866 2024-02-07T12:22:44.241 INFO:teuthology.orchestra.run.smithi078.stdout:34359738390 2024-02-07T12:22:44.584 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:44 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/2983159635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-07T12:22:44.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:44 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:44.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:44 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:44.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:44 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:22:44.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:44 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:44.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:44 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:44.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:44 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:22:44.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:44 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3494090052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-07T12:22:44.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:44 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/2983159635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-07T12:22:44.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:44 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:44.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:44 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:44.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:44 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:22:44.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:44 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:44.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:44 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:44.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:44 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:22:44.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:44 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/3494090052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-07T12:22:44.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:44 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/2983159635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-07T12:22:44.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:44 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:44.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:44 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:44.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:44 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T12:22:44.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:44 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:44.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:44 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T12:22:44.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:44 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T12:22:44.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:44 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/3494090052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-07T12:22:44.943 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574865 got 55834574866 for osd.1 2024-02-07T12:22:44.943 DEBUG:teuthology.parallel:result is None 2024-02-07T12:22:45.369 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738389 got 34359738390 for osd.0 2024-02-07T12:22:45.369 DEBUG:teuthology.parallel:result is None 2024-02-07T12:22:45.369 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-07T12:22:45.369 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-07T12:22:45.625 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: Adjusting osd_memory_target on smithi078 to 16064M 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: Updating smithi106:/etc/ceph/ceph.conf 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: Updating smithi171:/etc/ceph/ceph.conf 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:45 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-07T12:22:45.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: Adjusting osd_memory_target on smithi078 to 16064M 2024-02-07T12:22:45.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-07T12:22:45.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: Updating smithi106:/etc/ceph/ceph.conf 2024-02-07T12:22:45.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: Updating smithi171:/etc/ceph/ceph.conf 2024-02-07T12:22:45.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:22:45.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:22:45.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:22:45.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:45 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-07T12:22:45.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: Adjusting osd_memory_target on smithi078 to 16064M 2024-02-07T12:22:45.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-07T12:22:45.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: Updating smithi106:/etc/ceph/ceph.conf 2024-02-07T12:22:45.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: Updating smithi171:/etc/ceph/ceph.conf 2024-02-07T12:22:45.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:22:45.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:22:45.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T12:22:45.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:45.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:45 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:46.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:46 smithi171 ceph-mon[30005]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:22:46.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:46 smithi171 ceph-mon[30005]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:22:46.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:46 smithi171 ceph-mon[30005]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:22:46.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:46 smithi171 ceph-mon[30005]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:22:46.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:46 smithi171 ceph-mon[30005]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:22:46.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:46 smithi171 ceph-mon[30005]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:22:46.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:46 smithi171 ceph-mon[30005]: Deploying daemon alertmanager.a on smithi078 2024-02-07T12:22:46.730 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:46 smithi078 ceph-mon[25576]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:22:46.730 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:46 smithi078 ceph-mon[25576]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:22:46.730 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:46 smithi078 ceph-mon[25576]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:22:46.730 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:46 smithi078 ceph-mon[25576]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:22:46.730 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:46 smithi078 ceph-mon[25576]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:22:46.730 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:46 smithi078 ceph-mon[25576]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:22:46.731 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:46 smithi078 ceph-mon[25576]: Deploying daemon alertmanager.a on smithi078 2024-02-07T12:22:46.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:46 smithi106 ceph-mon[29693]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:22:46.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:46 smithi106 ceph-mon[29693]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:22:46.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:46 smithi106 ceph-mon[29693]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-02-07T12:22:46.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:46 smithi106 ceph-mon[29693]: Updating smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:22:46.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:46 smithi106 ceph-mon[29693]: Updating smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:22:46.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:46 smithi106 ceph-mon[29693]: Updating smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T12:22:46.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:46 smithi106 ceph-mon[29693]: Deploying daemon alertmanager.a on smithi078 2024-02-07T12:22:47.438 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:22:47.440 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-02-07T12:22:47.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:47 smithi171 ceph-mon[30005]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-07T12:22:47.678 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:47 smithi106 ceph-mon[29693]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-07T12:22:47.880 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:47 smithi078 ceph-mon[25576]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-07T12:22:48.426 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-02-07T12:22:45.463156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96612,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126556,"statfs":{"total":287972524032,"available":287873593344,"internally_reserved":0,"allocated":3158016,"data_stored":2100177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4783,"internal_metadata":95743313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.000355"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451169+0000","last_change":"2024-02-07T12:22:11.481050+0000","last_active":"2024-02-07T12:22:33.451169+0000","last_peered":"2024-02-07T12:22:33.451169+0000","last_clean":"2024-02-07T12:22:33.451169+0000","last_became_active":"2024-02-07T12:22:11.480384+0000","last_became_peered":"2024-02-07T12:22:11.480384+0000","last_unstale":"2024-02-07T12:22:33.451169+0000","last_undegraded":"2024-02-07T12:22:33.451169+0000","last_fullsized":"2024-02-07T12:22:33.451169+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:51:48.869912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454450+0000","last_change":"2024-02-07T12:22:09.463584+0000","last_active":"2024-02-07T12:22:33.454450+0000","last_peered":"2024-02-07T12:22:33.454450+0000","last_clean":"2024-02-07T12:22:33.454450+0000","last_became_active":"2024-02-07T12:22:09.462997+0000","last_became_peered":"2024-02-07T12:22:09.462997+0000","last_unstale":"2024-02-07T12:22:33.454450+0000","last_undegraded":"2024-02-07T12:22:33.454450+0000","last_fullsized":"2024-02-07T12:22:33.454450+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:40:58.176999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453676+0000","last_change":"2024-02-07T12:22:05.437143+0000","last_active":"2024-02-07T12:22:33.453676+0000","last_peered":"2024-02-07T12:22:33.453676+0000","last_clean":"2024-02-07T12:22:33.453676+0000","last_became_active":"2024-02-07T12:22:05.436561+0000","last_became_peered":"2024-02-07T12:22:05.436561+0000","last_unstale":"2024-02-07T12:22:33.453676+0000","last_undegraded":"2024-02-07T12:22:33.453676+0000","last_fullsized":"2024-02-07T12:22:33.453676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:51:44.462007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452797+0000","last_change":"2024-02-07T12:22:07.449909+0000","last_active":"2024-02-07T12:22:33.452797+0000","last_peered":"2024-02-07T12:22:33.452797+0000","last_clean":"2024-02-07T12:22:33.452797+0000","last_became_active":"2024-02-07T12:22:07.449549+0000","last_became_peered":"2024-02-07T12:22:07.449549+0000","last_unstale":"2024-02-07T12:22:33.452797+0000","last_undegraded":"2024-02-07T12:22:33.452797+0000","last_fullsized":"2024-02-07T12:22:33.452797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:55:40.361392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450157+0000","last_change":"2024-02-07T12:22:11.485021+0000","last_active":"2024-02-07T12:22:33.450157+0000","last_peered":"2024-02-07T12:22:33.450157+0000","last_clean":"2024-02-07T12:22:33.450157+0000","last_became_active":"2024-02-07T12:22:11.484690+0000","last_became_peered":"2024-02-07T12:22:11.484690+0000","last_unstale":"2024-02-07T12:22:33.450157+0000","last_undegraded":"2024-02-07T12:22:33.450157+0000","last_fullsized":"2024-02-07T12:22:33.450157+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:33:06.201071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451022+0000","last_change":"2024-02-07T12:22:09.462919+0000","last_active":"2024-02-07T12:22:33.451022+0000","last_peered":"2024-02-07T12:22:33.451022+0000","last_clean":"2024-02-07T12:22:33.451022+0000","last_became_active":"2024-02-07T12:22:09.462545+0000","last_became_peered":"2024-02-07T12:22:09.462545+0000","last_unstale":"2024-02-07T12:22:33.451022+0000","last_undegraded":"2024-02-07T12:22:33.451022+0000","last_fullsized":"2024-02-07T12:22:33.451022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:38:59.919191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453009+0000","last_change":"2024-02-07T12:22:05.437695+0000","last_active":"2024-02-07T12:22:33.453009+0000","last_peered":"2024-02-07T12:22:33.453009+0000","last_clean":"2024-02-07T12:22:33.453009+0000","last_became_active":"2024-02-07T12:22:05.437395+0000","last_became_peered":"2024-02-07T12:22:05.437395+0000","last_unstale":"2024-02-07T12:22:33.453009+0000","last_undegraded":"2024-02-07T12:22:33.453009+0000","last_fullsized":"2024-02-07T12:22:33.453009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:30:28.867141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453452+0000","last_change":"2024-02-07T12:22:07.443515+0000","last_active":"2024-02-07T12:22:33.453452+0000","last_peered":"2024-02-07T12:22:33.453452+0000","last_clean":"2024-02-07T12:22:33.453452+0000","last_became_active":"2024-02-07T12:22:07.443037+0000","last_became_peered":"2024-02-07T12:22:07.443037+0000","last_unstale":"2024-02-07T12:22:33.453452+0000","last_undegraded":"2024-02-07T12:22:33.453452+0000","last_fullsized":"2024-02-07T12:22:33.453452+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:07:36.609536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455367+0000","last_change":"2024-02-07T12:22:11.475782+0000","last_active":"2024-02-07T12:22:33.455367+0000","last_peered":"2024-02-07T12:22:33.455367+0000","last_clean":"2024-02-07T12:22:33.455367+0000","last_became_active":"2024-02-07T12:22:11.475487+0000","last_became_peered":"2024-02-07T12:22:11.475487+0000","last_unstale":"2024-02-07T12:22:33.455367+0000","last_undegraded":"2024-02-07T12:22:33.455367+0000","last_fullsized":"2024-02-07T12:22:33.455367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:34:11.990286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451738+0000","last_change":"2024-02-07T12:22:09.463619+0000","last_active":"2024-02-07T12:22:33.451738+0000","last_peered":"2024-02-07T12:22:33.451738+0000","last_clean":"2024-02-07T12:22:33.451738+0000","last_became_active":"2024-02-07T12:22:09.463245+0000","last_became_peered":"2024-02-07T12:22:09.463245+0000","last_unstale":"2024-02-07T12:22:33.451738+0000","last_undegraded":"2024-02-07T12:22:33.451738+0000","last_fullsized":"2024-02-07T12:22:33.451738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:57:37.634746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.457211+0000","last_change":"2024-02-07T12:22:05.434723+0000","last_active":"2024-02-07T12:22:33.457211+0000","last_peered":"2024-02-07T12:22:33.457211+0000","last_clean":"2024-02-07T12:22:33.457211+0000","last_became_active":"2024-02-07T12:22:05.431841+0000","last_became_peered":"2024-02-07T12:22:05.431841+0000","last_unstale":"2024-02-07T12:22:33.457211+0000","last_undegraded":"2024-02-07T12:22:33.457211+0000","last_fullsized":"2024-02-07T12:22:33.457211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:33:56.981715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451466+0000","last_change":"2024-02-07T12:22:07.448254+0000","last_active":"2024-02-07T12:22:33.451466+0000","last_peered":"2024-02-07T12:22:33.451466+0000","last_clean":"2024-02-07T12:22:33.451466+0000","last_became_active":"2024-02-07T12:22:07.447711+0000","last_became_peered":"2024-02-07T12:22:07.447711+0000","last_unstale":"2024-02-07T12:22:33.451466+0000","last_undegraded":"2024-02-07T12:22:33.451466+0000","last_fullsized":"2024-02-07T12:22:33.451466+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:10:28.271712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450782+0000","last_change":"2024-02-07T12:22:11.483371+0000","last_active":"2024-02-07T12:22:33.450782+0000","last_peered":"2024-02-07T12:22:33.450782+0000","last_clean":"2024-02-07T12:22:33.450782+0000","last_became_active":"2024-02-07T12:22:11.481464+0000","last_became_peered":"2024-02-07T12:22:11.481464+0000","last_unstale":"2024-02-07T12:22:33.450782+0000","last_undegraded":"2024-02-07T12:22:33.450782+0000","last_fullsized":"2024-02-07T12:22:33.450782+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:42:16.657815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451163+0000","last_change":"2024-02-07T12:22:09.462401+0000","last_active":"2024-02-07T12:22:33.451163+0000","last_peered":"2024-02-07T12:22:33.451163+0000","last_clean":"2024-02-07T12:22:33.451163+0000","last_became_active":"2024-02-07T12:22:09.462083+0000","last_became_peered":"2024-02-07T12:22:09.462083+0000","last_unstale":"2024-02-07T12:22:33.451163+0000","last_undegraded":"2024-02-07T12:22:33.451163+0000","last_fullsized":"2024-02-07T12:22:33.451163+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:17:52.651864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455414+0000","last_change":"2024-02-07T12:22:05.432478+0000","last_active":"2024-02-07T12:22:33.455414+0000","last_peered":"2024-02-07T12:22:33.455414+0000","last_clean":"2024-02-07T12:22:33.455414+0000","last_became_active":"2024-02-07T12:22:05.431921+0000","last_became_peered":"2024-02-07T12:22:05.431921+0000","last_unstale":"2024-02-07T12:22:33.455414+0000","last_undegraded":"2024-02-07T12:22:33.455414+0000","last_fullsized":"2024-02-07T12:22:33.455414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:48:08.676509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.457156+0000","last_change":"2024-02-07T12:22:07.450762+0000","last_active":"2024-02-07T12:22:33.457156+0000","last_peered":"2024-02-07T12:22:33.457156+0000","last_clean":"2024-02-07T12:22:33.457156+0000","last_became_active":"2024-02-07T12:22:07.450420+0000","last_became_peered":"2024-02-07T12:22:07.450420+0000","last_unstale":"2024-02-07T12:22:33.457156+0000","last_undegraded":"2024-02-07T12:22:33.457156+0000","last_fullsized":"2024-02-07T12:22:33.457156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:59:46.913171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451191+0000","last_change":"2024-02-07T12:22:11.482105+0000","last_active":"2024-02-07T12:22:33.451191+0000","last_peered":"2024-02-07T12:22:33.451191+0000","last_clean":"2024-02-07T12:22:33.451191+0000","last_became_active":"2024-02-07T12:22:11.481719+0000","last_became_peered":"2024-02-07T12:22:11.481719+0000","last_unstale":"2024-02-07T12:22:33.451191+0000","last_undegraded":"2024-02-07T12:22:33.451191+0000","last_fullsized":"2024-02-07T12:22:33.451191+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:02:37.188006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455844+0000","last_change":"2024-02-07T12:22:09.464830+0000","last_active":"2024-02-07T12:22:33.455844+0000","last_peered":"2024-02-07T12:22:33.455844+0000","last_clean":"2024-02-07T12:22:33.455844+0000","last_became_active":"2024-02-07T12:22:09.463802+0000","last_became_peered":"2024-02-07T12:22:09.463802+0000","last_unstale":"2024-02-07T12:22:33.455844+0000","last_undegraded":"2024-02-07T12:22:33.455844+0000","last_fullsized":"2024-02-07T12:22:33.455844+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:32:17.465677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456850+0000","last_change":"2024-02-07T12:22:05.434627+0000","last_active":"2024-02-07T12:22:33.456850+0000","last_peered":"2024-02-07T12:22:33.456850+0000","last_clean":"2024-02-07T12:22:33.456850+0000","last_became_active":"2024-02-07T12:22:05.432882+0000","last_became_peered":"2024-02-07T12:22:05.432882+0000","last_unstale":"2024-02-07T12:22:33.456850+0000","last_undegraded":"2024-02-07T12:22:33.456850+0000","last_fullsized":"2024-02-07T12:22:33.456850+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:59:52.006199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456030+0000","last_change":"2024-02-07T12:22:07.449956+0000","last_active":"2024-02-07T12:22:33.456030+0000","last_peered":"2024-02-07T12:22:33.456030+0000","last_clean":"2024-02-07T12:22:33.456030+0000","last_became_active":"2024-02-07T12:22:07.449683+0000","last_became_peered":"2024-02-07T12:22:07.449683+0000","last_unstale":"2024-02-07T12:22:33.456030+0000","last_undegraded":"2024-02-07T12:22:33.456030+0000","last_fullsized":"2024-02-07T12:22:33.456030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:58:12.419754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450683+0000","last_change":"2024-02-07T12:22:11.484506+0000","last_active":"2024-02-07T12:22:33.450683+0000","last_peered":"2024-02-07T12:22:33.450683+0000","last_clean":"2024-02-07T12:22:33.450683+0000","last_became_active":"2024-02-07T12:22:11.484115+0000","last_became_peered":"2024-02-07T12:22:11.484115+0000","last_unstale":"2024-02-07T12:22:33.450683+0000","last_undegraded":"2024-02-07T12:22:33.450683+0000","last_fullsized":"2024-02-07T12:22:33.450683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:07:35.410175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455303+0000","last_change":"2024-02-07T12:22:09.464581+0000","last_active":"2024-02-07T12:22:33.455303+0000","last_peered":"2024-02-07T12:22:33.455303+0000","last_clean":"2024-02-07T12:22:33.455303+0000","last_became_active":"2024-02-07T12:22:09.463821+0000","last_became_peered":"2024-02-07T12:22:09.463821+0000","last_unstale":"2024-02-07T12:22:33.455303+0000","last_undegraded":"2024-02-07T12:22:33.455303+0000","last_fullsized":"2024-02-07T12:22:33.455303+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:19:14.492671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451343+0000","last_change":"2024-02-07T12:22:05.437329+0000","last_active":"2024-02-07T12:22:33.451343+0000","last_peered":"2024-02-07T12:22:33.451343+0000","last_clean":"2024-02-07T12:22:33.451343+0000","last_became_active":"2024-02-07T12:22:05.437024+0000","last_became_peered":"2024-02-07T12:22:05.437024+0000","last_unstale":"2024-02-07T12:22:33.451343+0000","last_undegraded":"2024-02-07T12:22:33.451343+0000","last_fullsized":"2024-02-07T12:22:33.451343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:12:39.909665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456138+0000","last_change":"2024-02-07T12:22:07.451035+0000","last_active":"2024-02-07T12:22:33.456138+0000","last_peered":"2024-02-07T12:22:33.456138+0000","last_clean":"2024-02-07T12:22:33.456138+0000","last_became_active":"2024-02-07T12:22:07.449672+0000","last_became_peered":"2024-02-07T12:22:07.449672+0000","last_unstale":"2024-02-07T12:22:33.456138+0000","last_undegraded":"2024-02-07T12:22:33.456138+0000","last_fullsized":"2024-02-07T12:22:33.456138+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:27:09.879939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452117+0000","last_change":"2024-02-07T12:22:11.484559+0000","last_active":"2024-02-07T12:22:33.452117+0000","last_peered":"2024-02-07T12:22:33.452117+0000","last_clean":"2024-02-07T12:22:33.452117+0000","last_became_active":"2024-02-07T12:22:11.483907+0000","last_became_peered":"2024-02-07T12:22:11.483907+0000","last_unstale":"2024-02-07T12:22:33.452117+0000","last_undegraded":"2024-02-07T12:22:33.452117+0000","last_fullsized":"2024-02-07T12:22:33.452117+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:59:06.096473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478453+0000","last_change":"2024-02-07T12:22:09.465726+0000","last_active":"2024-02-07T12:22:35.478453+0000","last_peered":"2024-02-07T12:22:35.478453+0000","last_clean":"2024-02-07T12:22:35.478453+0000","last_became_active":"2024-02-07T12:22:09.464438+0000","last_became_peered":"2024-02-07T12:22:09.464438+0000","last_unstale":"2024-02-07T12:22:35.478453+0000","last_undegraded":"2024-02-07T12:22:35.478453+0000","last_fullsized":"2024-02-07T12:22:35.478453+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:37:20.915814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450384+0000","last_change":"2024-02-07T12:22:05.437758+0000","last_active":"2024-02-07T12:22:33.450384+0000","last_peered":"2024-02-07T12:22:33.450384+0000","last_clean":"2024-02-07T12:22:33.450384+0000","last_became_active":"2024-02-07T12:22:05.437453+0000","last_became_peered":"2024-02-07T12:22:05.437453+0000","last_unstale":"2024-02-07T12:22:33.450384+0000","last_undegraded":"2024-02-07T12:22:33.450384+0000","last_fullsized":"2024-02-07T12:22:33.450384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:37:24.386646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449457+0000","last_change":"2024-02-07T12:22:07.444158+0000","last_active":"2024-02-07T12:22:33.449457+0000","last_peered":"2024-02-07T12:22:33.449457+0000","last_clean":"2024-02-07T12:22:33.449457+0000","last_became_active":"2024-02-07T12:22:07.443846+0000","last_became_peered":"2024-02-07T12:22:07.443846+0000","last_unstale":"2024-02-07T12:22:33.449457+0000","last_undegraded":"2024-02-07T12:22:33.449457+0000","last_fullsized":"2024-02-07T12:22:33.449457+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:57:16.306199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451571+0000","last_change":"2024-02-07T12:22:11.483239+0000","last_active":"2024-02-07T12:22:33.451571+0000","last_peered":"2024-02-07T12:22:33.451571+0000","last_clean":"2024-02-07T12:22:33.451571+0000","last_became_active":"2024-02-07T12:22:11.481813+0000","last_became_peered":"2024-02-07T12:22:11.481813+0000","last_unstale":"2024-02-07T12:22:33.451571+0000","last_undegraded":"2024-02-07T12:22:33.451571+0000","last_fullsized":"2024-02-07T12:22:33.451571+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:25:35.175522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452288+0000","last_change":"2024-02-07T12:22:09.466309+0000","last_active":"2024-02-07T12:22:33.452288+0000","last_peered":"2024-02-07T12:22:33.452288+0000","last_clean":"2024-02-07T12:22:33.452288+0000","last_became_active":"2024-02-07T12:22:09.465935+0000","last_became_peered":"2024-02-07T12:22:09.465935+0000","last_unstale":"2024-02-07T12:22:33.452288+0000","last_undegraded":"2024-02-07T12:22:33.452288+0000","last_fullsized":"2024-02-07T12:22:33.452288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:52:32.912644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454510+0000","last_change":"2024-02-07T12:22:05.434566+0000","last_active":"2024-02-07T12:22:33.454510+0000","last_peered":"2024-02-07T12:22:33.454510+0000","last_clean":"2024-02-07T12:22:33.454510+0000","last_became_active":"2024-02-07T12:22:05.432441+0000","last_became_peered":"2024-02-07T12:22:05.432441+0000","last_unstale":"2024-02-07T12:22:33.454510+0000","last_undegraded":"2024-02-07T12:22:33.454510+0000","last_fullsized":"2024-02-07T12:22:33.454510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:39:13.404359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449693+0000","last_change":"2024-02-07T12:22:07.444290+0000","last_active":"2024-02-07T12:22:33.449693+0000","last_peered":"2024-02-07T12:22:33.449693+0000","last_clean":"2024-02-07T12:22:33.449693+0000","last_became_active":"2024-02-07T12:22:07.443906+0000","last_became_peered":"2024-02-07T12:22:07.443906+0000","last_unstale":"2024-02-07T12:22:33.449693+0000","last_undegraded":"2024-02-07T12:22:33.449693+0000","last_fullsized":"2024-02-07T12:22:33.449693+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:28:57.762165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451451+0000","last_change":"2024-02-07T12:22:11.480678+0000","last_active":"2024-02-07T12:22:33.451451+0000","last_peered":"2024-02-07T12:22:33.451451+0000","last_clean":"2024-02-07T12:22:33.451451+0000","last_became_active":"2024-02-07T12:22:11.479936+0000","last_became_peered":"2024-02-07T12:22:11.479936+0000","last_unstale":"2024-02-07T12:22:33.451451+0000","last_undegraded":"2024-02-07T12:22:33.451451+0000","last_fullsized":"2024-02-07T12:22:33.451451+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:21:27.000037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449584+0000","last_change":"2024-02-07T12:22:09.464966+0000","last_active":"2024-02-07T12:22:33.449584+0000","last_peered":"2024-02-07T12:22:33.449584+0000","last_clean":"2024-02-07T12:22:33.449584+0000","last_became_active":"2024-02-07T12:22:09.464491+0000","last_became_peered":"2024-02-07T12:22:09.464491+0000","last_unstale":"2024-02-07T12:22:33.449584+0000","last_undegraded":"2024-02-07T12:22:33.449584+0000","last_fullsized":"2024-02-07T12:22:33.449584+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:48:10.264312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453212+0000","last_change":"2024-02-07T12:22:05.432383+0000","last_active":"2024-02-07T12:22:33.453212+0000","last_peered":"2024-02-07T12:22:33.453212+0000","last_clean":"2024-02-07T12:22:33.453212+0000","last_became_active":"2024-02-07T12:22:05.432040+0000","last_became_peered":"2024-02-07T12:22:05.432040+0000","last_unstale":"2024-02-07T12:22:33.453212+0000","last_undegraded":"2024-02-07T12:22:33.453212+0000","last_fullsized":"2024-02-07T12:22:33.453212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:48:34.693979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452737+0000","last_change":"2024-02-07T12:22:07.441536+0000","last_active":"2024-02-07T12:22:33.452737+0000","last_peered":"2024-02-07T12:22:33.452737+0000","last_clean":"2024-02-07T12:22:33.452737+0000","last_became_active":"2024-02-07T12:22:07.441247+0000","last_became_peered":"2024-02-07T12:22:07.441247+0000","last_unstale":"2024-02-07T12:22:33.452737+0000","last_undegraded":"2024-02-07T12:22:33.452737+0000","last_fullsized":"2024-02-07T12:22:33.452737+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:24:17.950707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450857+0000","last_change":"2024-02-07T12:22:11.481427+0000","last_active":"2024-02-07T12:22:33.450857+0000","last_peered":"2024-02-07T12:22:33.450857+0000","last_clean":"2024-02-07T12:22:33.450857+0000","last_became_active":"2024-02-07T12:22:11.479878+0000","last_became_peered":"2024-02-07T12:22:11.479878+0000","last_unstale":"2024-02-07T12:22:33.450857+0000","last_undegraded":"2024-02-07T12:22:33.450857+0000","last_fullsized":"2024-02-07T12:22:33.450857+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:03:02.219923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449834+0000","last_change":"2024-02-07T12:22:09.467738+0000","last_active":"2024-02-07T12:22:33.449834+0000","last_peered":"2024-02-07T12:22:33.449834+0000","last_clean":"2024-02-07T12:22:33.449834+0000","last_became_active":"2024-02-07T12:22:09.467436+0000","last_became_peered":"2024-02-07T12:22:09.467436+0000","last_unstale":"2024-02-07T12:22:33.449834+0000","last_undegraded":"2024-02-07T12:22:33.449834+0000","last_fullsized":"2024-02-07T12:22:33.449834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:15:36.041735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452581+0000","last_change":"2024-02-07T12:22:05.434184+0000","last_active":"2024-02-07T12:22:33.452581+0000","last_peered":"2024-02-07T12:22:33.452581+0000","last_clean":"2024-02-07T12:22:33.452581+0000","last_became_active":"2024-02-07T12:22:05.433739+0000","last_became_peered":"2024-02-07T12:22:05.433739+0000","last_unstale":"2024-02-07T12:22:33.452581+0000","last_undegraded":"2024-02-07T12:22:33.452581+0000","last_fullsized":"2024-02-07T12:22:33.452581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:31:11.589928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453574+0000","last_change":"2024-02-07T12:22:07.446985+0000","last_active":"2024-02-07T12:22:33.453574+0000","last_peered":"2024-02-07T12:22:33.453574+0000","last_clean":"2024-02-07T12:22:33.453574+0000","last_became_active":"2024-02-07T12:22:07.446529+0000","last_became_peered":"2024-02-07T12:22:07.446529+0000","last_unstale":"2024-02-07T12:22:33.453574+0000","last_undegraded":"2024-02-07T12:22:33.453574+0000","last_fullsized":"2024-02-07T12:22:33.453574+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:52:48.183777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455466+0000","last_change":"2024-02-07T12:22:11.478689+0000","last_active":"2024-02-07T12:22:33.455466+0000","last_peered":"2024-02-07T12:22:33.455466+0000","last_clean":"2024-02-07T12:22:33.455466+0000","last_became_active":"2024-02-07T12:22:11.478405+0000","last_became_peered":"2024-02-07T12:22:11.478405+0000","last_unstale":"2024-02-07T12:22:33.455466+0000","last_undegraded":"2024-02-07T12:22:33.455466+0000","last_fullsized":"2024-02-07T12:22:33.455466+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:28:16.251286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451336+0000","last_change":"2024-02-07T12:22:09.462699+0000","last_active":"2024-02-07T12:22:33.451336+0000","last_peered":"2024-02-07T12:22:33.451336+0000","last_clean":"2024-02-07T12:22:33.451336+0000","last_became_active":"2024-02-07T12:22:09.462164+0000","last_became_peered":"2024-02-07T12:22:09.462164+0000","last_unstale":"2024-02-07T12:22:33.451336+0000","last_undegraded":"2024-02-07T12:22:33.451336+0000","last_fullsized":"2024-02-07T12:22:33.451336+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:46:53.340951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.457110+0000","last_change":"2024-02-07T12:22:05.436026+0000","last_active":"2024-02-07T12:22:33.457110+0000","last_peered":"2024-02-07T12:22:33.457110+0000","last_clean":"2024-02-07T12:22:33.457110+0000","last_became_active":"2024-02-07T12:22:05.434573+0000","last_became_peered":"2024-02-07T12:22:05.434573+0000","last_unstale":"2024-02-07T12:22:33.457110+0000","last_undegraded":"2024-02-07T12:22:33.457110+0000","last_fullsized":"2024-02-07T12:22:33.457110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:52:17.391908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451619+0000","last_change":"2024-02-07T12:22:07.447201+0000","last_active":"2024-02-07T12:22:33.451619+0000","last_peered":"2024-02-07T12:22:33.451619+0000","last_clean":"2024-02-07T12:22:33.451619+0000","last_became_active":"2024-02-07T12:22:07.445745+0000","last_became_peered":"2024-02-07T12:22:07.445745+0000","last_unstale":"2024-02-07T12:22:33.451619+0000","last_undegraded":"2024-02-07T12:22:33.451619+0000","last_fullsized":"2024-02-07T12:22:33.451619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:40:13.014137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454837+0000","last_change":"2024-02-07T12:22:11.475394+0000","last_active":"2024-02-07T12:22:33.454837+0000","last_peered":"2024-02-07T12:22:33.454837+0000","last_clean":"2024-02-07T12:22:33.454837+0000","last_became_active":"2024-02-07T12:22:11.475099+0000","last_became_peered":"2024-02-07T12:22:11.475099+0000","last_unstale":"2024-02-07T12:22:33.454837+0000","last_undegraded":"2024-02-07T12:22:33.454837+0000","last_fullsized":"2024-02-07T12:22:33.454837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:11:42.390618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450116+0000","last_change":"2024-02-07T12:22:09.465490+0000","last_active":"2024-02-07T12:22:33.450116+0000","last_peered":"2024-02-07T12:22:33.450116+0000","last_clean":"2024-02-07T12:22:33.450116+0000","last_became_active":"2024-02-07T12:22:09.465079+0000","last_became_peered":"2024-02-07T12:22:09.465079+0000","last_unstale":"2024-02-07T12:22:33.450116+0000","last_undegraded":"2024-02-07T12:22:33.450116+0000","last_fullsized":"2024-02-07T12:22:33.450116+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:55:18.079376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451367+0000","last_change":"2024-02-07T12:22:05.435525+0000","last_active":"2024-02-07T12:22:33.451367+0000","last_peered":"2024-02-07T12:22:33.451367+0000","last_clean":"2024-02-07T12:22:33.451367+0000","last_became_active":"2024-02-07T12:22:05.435110+0000","last_became_peered":"2024-02-07T12:22:05.435110+0000","last_unstale":"2024-02-07T12:22:33.451367+0000","last_undegraded":"2024-02-07T12:22:33.451367+0000","last_fullsized":"2024-02-07T12:22:33.451367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:43:03.793348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456941+0000","last_change":"2024-02-07T12:22:07.450539+0000","last_active":"2024-02-07T12:22:33.456941+0000","last_peered":"2024-02-07T12:22:33.456941+0000","last_clean":"2024-02-07T12:22:33.456941+0000","last_became_active":"2024-02-07T12:22:07.450216+0000","last_became_peered":"2024-02-07T12:22:07.450216+0000","last_unstale":"2024-02-07T12:22:33.456941+0000","last_undegraded":"2024-02-07T12:22:33.456941+0000","last_fullsized":"2024-02-07T12:22:33.456941+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:14:33.069337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451879+0000","last_change":"2024-02-07T12:22:11.482109+0000","last_active":"2024-02-07T12:22:33.451879+0000","last_peered":"2024-02-07T12:22:33.451879+0000","last_clean":"2024-02-07T12:22:33.451879+0000","last_became_active":"2024-02-07T12:22:11.481555+0000","last_became_peered":"2024-02-07T12:22:11.481555+0000","last_unstale":"2024-02-07T12:22:33.451879+0000","last_undegraded":"2024-02-07T12:22:33.451879+0000","last_fullsized":"2024-02-07T12:22:33.451879+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:26:49.826771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478377+0000","last_change":"2024-02-07T12:22:09.464962+0000","last_active":"2024-02-07T12:22:35.478377+0000","last_peered":"2024-02-07T12:22:35.478377+0000","last_clean":"2024-02-07T12:22:35.478377+0000","last_became_active":"2024-02-07T12:22:09.464433+0000","last_became_peered":"2024-02-07T12:22:09.464433+0000","last_unstale":"2024-02-07T12:22:35.478377+0000","last_undegraded":"2024-02-07T12:22:35.478377+0000","last_fullsized":"2024-02-07T12:22:35.478377+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:08:52.172867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455940+0000","last_change":"2024-02-07T12:22:05.431234+0000","last_active":"2024-02-07T12:22:33.455940+0000","last_peered":"2024-02-07T12:22:33.455940+0000","last_clean":"2024-02-07T12:22:33.455940+0000","last_became_active":"2024-02-07T12:22:05.430815+0000","last_became_peered":"2024-02-07T12:22:05.430815+0000","last_unstale":"2024-02-07T12:22:33.455940+0000","last_undegraded":"2024-02-07T12:22:33.455940+0000","last_fullsized":"2024-02-07T12:22:33.455940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:34:58.630000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450373+0000","last_change":"2024-02-07T12:22:07.443330+0000","last_active":"2024-02-07T12:22:33.450373+0000","last_peered":"2024-02-07T12:22:33.450373+0000","last_clean":"2024-02-07T12:22:33.450373+0000","last_became_active":"2024-02-07T12:22:07.442911+0000","last_became_peered":"2024-02-07T12:22:07.442911+0000","last_unstale":"2024-02-07T12:22:33.450373+0000","last_undegraded":"2024-02-07T12:22:33.450373+0000","last_fullsized":"2024-02-07T12:22:33.450373+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:53:22.141512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451750+0000","last_change":"2024-02-07T12:22:11.481988+0000","last_active":"2024-02-07T12:22:33.451750+0000","last_peered":"2024-02-07T12:22:33.451750+0000","last_clean":"2024-02-07T12:22:33.451750+0000","last_became_active":"2024-02-07T12:22:11.481344+0000","last_became_peered":"2024-02-07T12:22:11.481344+0000","last_unstale":"2024-02-07T12:22:33.451750+0000","last_undegraded":"2024-02-07T12:22:33.451750+0000","last_fullsized":"2024-02-07T12:22:33.451750+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:34:34.290136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478466+0000","last_change":"2024-02-07T12:22:09.462773+0000","last_active":"2024-02-07T12:22:35.478466+0000","last_peered":"2024-02-07T12:22:35.478466+0000","last_clean":"2024-02-07T12:22:35.478466+0000","last_became_active":"2024-02-07T12:22:09.462241+0000","last_became_peered":"2024-02-07T12:22:09.462241+0000","last_unstale":"2024-02-07T12:22:35.478466+0000","last_undegraded":"2024-02-07T12:22:35.478466+0000","last_fullsized":"2024-02-07T12:22:35.478466+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:30:08.899685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455622+0000","last_change":"2024-02-07T12:22:05.435004+0000","last_active":"2024-02-07T12:22:33.455622+0000","last_peered":"2024-02-07T12:22:33.455622+0000","last_clean":"2024-02-07T12:22:33.455622+0000","last_became_active":"2024-02-07T12:22:05.432534+0000","last_became_peered":"2024-02-07T12:22:05.432534+0000","last_unstale":"2024-02-07T12:22:33.455622+0000","last_undegraded":"2024-02-07T12:22:33.455622+0000","last_fullsized":"2024-02-07T12:22:33.455622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:14:06.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451565+0000","last_change":"2024-02-07T12:22:07.446909+0000","last_active":"2024-02-07T12:22:33.451565+0000","last_peered":"2024-02-07T12:22:33.451565+0000","last_clean":"2024-02-07T12:22:33.451565+0000","last_became_active":"2024-02-07T12:22:07.445728+0000","last_became_peered":"2024-02-07T12:22:07.445728+0000","last_unstale":"2024-02-07T12:22:33.451565+0000","last_undegraded":"2024-02-07T12:22:33.451565+0000","last_fullsized":"2024-02-07T12:22:33.451565+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:31:36.456778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451257+0000","last_change":"2024-02-07T12:22:11.483311+0000","last_active":"2024-02-07T12:22:33.451257+0000","last_peered":"2024-02-07T12:22:33.451257+0000","last_clean":"2024-02-07T12:22:33.451257+0000","last_became_active":"2024-02-07T12:22:11.482210+0000","last_became_peered":"2024-02-07T12:22:11.482210+0000","last_unstale":"2024-02-07T12:22:33.451257+0000","last_undegraded":"2024-02-07T12:22:33.451257+0000","last_fullsized":"2024-02-07T12:22:33.451257+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:07:07.948765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453319+0000","last_change":"2024-02-07T12:22:09.465917+0000","last_active":"2024-02-07T12:22:33.453319+0000","last_peered":"2024-02-07T12:22:33.453319+0000","last_clean":"2024-02-07T12:22:33.453319+0000","last_became_active":"2024-02-07T12:22:09.465596+0000","last_became_peered":"2024-02-07T12:22:09.465596+0000","last_unstale":"2024-02-07T12:22:33.453319+0000","last_undegraded":"2024-02-07T12:22:33.453319+0000","last_fullsized":"2024-02-07T12:22:33.453319+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:35:08.970185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450619+0000","last_change":"2024-02-07T12:22:05.434175+0000","last_active":"2024-02-07T12:22:33.450619+0000","last_peered":"2024-02-07T12:22:33.450619+0000","last_clean":"2024-02-07T12:22:33.450619+0000","last_became_active":"2024-02-07T12:22:05.433784+0000","last_became_peered":"2024-02-07T12:22:05.433784+0000","last_unstale":"2024-02-07T12:22:33.450619+0000","last_undegraded":"2024-02-07T12:22:33.450619+0000","last_fullsized":"2024-02-07T12:22:33.450619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:21:04.038244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450543+0000","last_change":"2024-02-07T12:22:07.448016+0000","last_active":"2024-02-07T12:22:33.450543+0000","last_peered":"2024-02-07T12:22:33.450543+0000","last_clean":"2024-02-07T12:22:33.450543+0000","last_became_active":"2024-02-07T12:22:07.446717+0000","last_became_peered":"2024-02-07T12:22:07.446717+0000","last_unstale":"2024-02-07T12:22:33.450543+0000","last_undegraded":"2024-02-07T12:22:33.450543+0000","last_fullsized":"2024-02-07T12:22:33.450543+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:08:09.019276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453079+0000","last_change":"2024-02-07T12:22:11.485136+0000","last_active":"2024-02-07T12:22:33.453079+0000","last_peered":"2024-02-07T12:22:33.453079+0000","last_clean":"2024-02-07T12:22:33.453079+0000","last_became_active":"2024-02-07T12:22:11.484737+0000","last_became_peered":"2024-02-07T12:22:11.484737+0000","last_unstale":"2024-02-07T12:22:33.453079+0000","last_undegraded":"2024-02-07T12:22:33.453079+0000","last_fullsized":"2024-02-07T12:22:33.453079+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:14:39.741747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452373+0000","last_change":"2024-02-07T12:22:09.466941+0000","last_active":"2024-02-07T12:22:33.452373+0000","last_peered":"2024-02-07T12:22:33.452373+0000","last_clean":"2024-02-07T12:22:33.452373+0000","last_became_active":"2024-02-07T12:22:09.466585+0000","last_became_peered":"2024-02-07T12:22:09.466585+0000","last_unstale":"2024-02-07T12:22:33.452373+0000","last_undegraded":"2024-02-07T12:22:33.452373+0000","last_fullsized":"2024-02-07T12:22:33.452373+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:42:26.039043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454938+0000","last_change":"2024-02-07T12:22:05.435052+0000","last_active":"2024-02-07T12:22:33.454938+0000","last_peered":"2024-02-07T12:22:33.454938+0000","last_clean":"2024-02-07T12:22:33.454938+0000","last_became_active":"2024-02-07T12:22:05.434724+0000","last_became_peered":"2024-02-07T12:22:05.434724+0000","last_unstale":"2024-02-07T12:22:33.454938+0000","last_undegraded":"2024-02-07T12:22:33.454938+0000","last_fullsized":"2024-02-07T12:22:33.454938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:18:08.397410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454959+0000","last_change":"2024-02-07T12:22:07.449850+0000","last_active":"2024-02-07T12:22:33.454959+0000","last_peered":"2024-02-07T12:22:33.454959+0000","last_clean":"2024-02-07T12:22:33.454959+0000","last_became_active":"2024-02-07T12:22:07.449562+0000","last_became_peered":"2024-02-07T12:22:07.449562+0000","last_unstale":"2024-02-07T12:22:33.454959+0000","last_undegraded":"2024-02-07T12:22:33.454959+0000","last_fullsized":"2024-02-07T12:22:33.454959+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:56:16.044010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455265+0000","last_change":"2024-02-07T12:22:11.476414+0000","last_active":"2024-02-07T12:22:33.455265+0000","last_peered":"2024-02-07T12:22:33.455265+0000","last_clean":"2024-02-07T12:22:33.455265+0000","last_became_active":"2024-02-07T12:22:11.476127+0000","last_became_peered":"2024-02-07T12:22:11.476127+0000","last_unstale":"2024-02-07T12:22:33.455265+0000","last_undegraded":"2024-02-07T12:22:33.455265+0000","last_fullsized":"2024-02-07T12:22:33.455265+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:53:27.539410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456982+0000","last_change":"2024-02-07T12:22:05.434014+0000","last_active":"2024-02-07T12:22:33.456982+0000","last_peered":"2024-02-07T12:22:33.456982+0000","last_clean":"2024-02-07T12:22:33.456982+0000","last_became_active":"2024-02-07T12:22:05.432346+0000","last_became_peered":"2024-02-07T12:22:05.432346+0000","last_unstale":"2024-02-07T12:22:33.456982+0000","last_undegraded":"2024-02-07T12:22:33.456982+0000","last_fullsized":"2024-02-07T12:22:33.456982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:53:46.026960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456176+0000","last_change":"2024-02-07T12:22:07.450695+0000","last_active":"2024-02-07T12:22:33.456176+0000","last_peered":"2024-02-07T12:22:33.456176+0000","last_clean":"2024-02-07T12:22:33.456176+0000","last_became_active":"2024-02-07T12:22:07.450276+0000","last_became_peered":"2024-02-07T12:22:07.450276+0000","last_unstale":"2024-02-07T12:22:33.456176+0000","last_undegraded":"2024-02-07T12:22:33.456176+0000","last_fullsized":"2024-02-07T12:22:33.456176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:15:25.118040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478534+0000","last_change":"2024-02-07T12:22:09.463241+0000","last_active":"2024-02-07T12:22:35.478534+0000","last_peered":"2024-02-07T12:22:35.478534+0000","last_clean":"2024-02-07T12:22:35.478534+0000","last_became_active":"2024-02-07T12:22:09.462959+0000","last_became_peered":"2024-02-07T12:22:09.462959+0000","last_unstale":"2024-02-07T12:22:35.478534+0000","last_undegraded":"2024-02-07T12:22:35.478534+0000","last_fullsized":"2024-02-07T12:22:35.478534+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:45:32.777028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450939+0000","last_change":"2024-02-07T12:22:11.481803+0000","last_active":"2024-02-07T12:22:33.450939+0000","last_peered":"2024-02-07T12:22:33.450939+0000","last_clean":"2024-02-07T12:22:33.450939+0000","last_became_active":"2024-02-07T12:22:11.481491+0000","last_became_peered":"2024-02-07T12:22:11.481491+0000","last_unstale":"2024-02-07T12:22:33.450939+0000","last_undegraded":"2024-02-07T12:22:33.450939+0000","last_fullsized":"2024-02-07T12:22:33.450939+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:52:47.766315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452629+0000","last_change":"2024-02-07T12:22:05.437055+0000","last_active":"2024-02-07T12:22:33.452629+0000","last_peered":"2024-02-07T12:22:33.452629+0000","last_clean":"2024-02-07T12:22:33.452629+0000","last_became_active":"2024-02-07T12:22:05.436596+0000","last_became_peered":"2024-02-07T12:22:05.436596+0000","last_unstale":"2024-02-07T12:22:33.452629+0000","last_undegraded":"2024-02-07T12:22:33.452629+0000","last_fullsized":"2024-02-07T12:22:33.452629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:05:04.000703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478419+0000","last_change":"2024-02-07T12:22:07.441869+0000","last_active":"2024-02-07T12:22:35.478419+0000","last_peered":"2024-02-07T12:22:35.478419+0000","last_clean":"2024-02-07T12:22:35.478419+0000","last_became_active":"2024-02-07T12:22:07.441411+0000","last_became_peered":"2024-02-07T12:22:07.441411+0000","last_unstale":"2024-02-07T12:22:35.478419+0000","last_undegraded":"2024-02-07T12:22:35.478419+0000","last_fullsized":"2024-02-07T12:22:35.478419+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:01:00.861008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455519+0000","last_change":"2024-02-07T12:22:09.462832+0000","last_active":"2024-02-07T12:22:33.455519+0000","last_peered":"2024-02-07T12:22:33.455519+0000","last_clean":"2024-02-07T12:22:33.455519+0000","last_became_active":"2024-02-07T12:22:09.462387+0000","last_became_peered":"2024-02-07T12:22:09.462387+0000","last_unstale":"2024-02-07T12:22:33.455519+0000","last_undegraded":"2024-02-07T12:22:33.455519+0000","last_fullsized":"2024-02-07T12:22:33.455519+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:06:13.802167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450203+0000","last_change":"2024-02-07T12:22:11.484556+0000","last_active":"2024-02-07T12:22:33.450203+0000","last_peered":"2024-02-07T12:22:33.450203+0000","last_clean":"2024-02-07T12:22:33.450203+0000","last_became_active":"2024-02-07T12:22:11.483965+0000","last_became_peered":"2024-02-07T12:22:11.483965+0000","last_unstale":"2024-02-07T12:22:33.450203+0000","last_undegraded":"2024-02-07T12:22:33.450203+0000","last_fullsized":"2024-02-07T12:22:33.450203+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:29:18.294341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451517+0000","last_change":"2024-02-07T12:22:05.432148+0000","last_active":"2024-02-07T12:22:33.451517+0000","last_peered":"2024-02-07T12:22:33.451517+0000","last_clean":"2024-02-07T12:22:33.451517+0000","last_became_active":"2024-02-07T12:22:05.431868+0000","last_became_peered":"2024-02-07T12:22:05.431868+0000","last_unstale":"2024-02-07T12:22:33.451517+0000","last_undegraded":"2024-02-07T12:22:33.451517+0000","last_fullsized":"2024-02-07T12:22:33.451517+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:00:56.788222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452432+0000","last_change":"2024-02-07T12:22:07.447135+0000","last_active":"2024-02-07T12:22:33.452432+0000","last_peered":"2024-02-07T12:22:33.452432+0000","last_clean":"2024-02-07T12:22:33.452432+0000","last_became_active":"2024-02-07T12:22:07.445819+0000","last_became_peered":"2024-02-07T12:22:07.445819+0000","last_unstale":"2024-02-07T12:22:33.452432+0000","last_undegraded":"2024-02-07T12:22:33.452432+0000","last_fullsized":"2024-02-07T12:22:33.452432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:46:40.686303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451968+0000","last_change":"2024-02-07T12:22:09.465184+0000","last_active":"2024-02-07T12:22:33.451968+0000","last_peered":"2024-02-07T12:22:33.451968+0000","last_clean":"2024-02-07T12:22:33.451968+0000","last_became_active":"2024-02-07T12:22:09.464618+0000","last_became_peered":"2024-02-07T12:22:09.464618+0000","last_unstale":"2024-02-07T12:22:33.451968+0000","last_undegraded":"2024-02-07T12:22:33.451968+0000","last_fullsized":"2024-02-07T12:22:33.451968+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:23:44.574942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452664+0000","last_change":"2024-02-07T12:22:11.485373+0000","last_active":"2024-02-07T12:22:33.452664+0000","last_peered":"2024-02-07T12:22:33.452664+0000","last_clean":"2024-02-07T12:22:33.452664+0000","last_became_active":"2024-02-07T12:22:11.485039+0000","last_became_peered":"2024-02-07T12:22:11.485039+0000","last_unstale":"2024-02-07T12:22:33.452664+0000","last_undegraded":"2024-02-07T12:22:33.452664+0000","last_fullsized":"2024-02-07T12:22:33.452664+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:25:47.014293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454260+0000","last_change":"2024-02-07T12:22:07.450879+0000","last_active":"2024-02-07T12:22:33.454260+0000","last_peered":"2024-02-07T12:22:33.454260+0000","last_clean":"2024-02-07T12:22:33.454260+0000","last_became_active":"2024-02-07T12:22:07.450404+0000","last_became_peered":"2024-02-07T12:22:07.450404+0000","last_unstale":"2024-02-07T12:22:33.454260+0000","last_undegraded":"2024-02-07T12:22:33.454260+0000","last_fullsized":"2024-02-07T12:22:33.454260+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:42:37.924581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449887+0000","last_change":"2024-02-07T12:22:05.436755+0000","last_active":"2024-02-07T12:22:33.449887+0000","last_peered":"2024-02-07T12:22:33.449887+0000","last_clean":"2024-02-07T12:22:33.449887+0000","last_became_active":"2024-02-07T12:22:05.436477+0000","last_became_peered":"2024-02-07T12:22:05.436477+0000","last_unstale":"2024-02-07T12:22:33.449887+0000","last_undegraded":"2024-02-07T12:22:33.449887+0000","last_fullsized":"2024-02-07T12:22:33.449887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:45:36.025351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453532+0000","last_change":"2024-02-07T12:22:09.468042+0000","last_active":"2024-02-07T12:22:33.453532+0000","last_peered":"2024-02-07T12:22:33.453532+0000","last_clean":"2024-02-07T12:22:33.453532+0000","last_became_active":"2024-02-07T12:22:09.467600+0000","last_became_peered":"2024-02-07T12:22:09.467600+0000","last_unstale":"2024-02-07T12:22:33.453532+0000","last_undegraded":"2024-02-07T12:22:33.453532+0000","last_fullsized":"2024-02-07T12:22:33.453532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:33:43.797109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454611+0000","last_change":"2024-02-07T12:22:11.478779+0000","last_active":"2024-02-07T12:22:33.454611+0000","last_peered":"2024-02-07T12:22:33.454611+0000","last_clean":"2024-02-07T12:22:33.454611+0000","last_became_active":"2024-02-07T12:22:11.478510+0000","last_became_peered":"2024-02-07T12:22:11.478510+0000","last_unstale":"2024-02-07T12:22:33.454611+0000","last_undegraded":"2024-02-07T12:22:33.454611+0000","last_fullsized":"2024-02-07T12:22:33.454611+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:40:27.319317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450259+0000","last_change":"2024-02-07T12:22:07.443955+0000","last_active":"2024-02-07T12:22:33.450259+0000","last_peered":"2024-02-07T12:22:33.450259+0000","last_clean":"2024-02-07T12:22:33.450259+0000","last_became_active":"2024-02-07T12:22:07.443467+0000","last_became_peered":"2024-02-07T12:22:07.443467+0000","last_unstale":"2024-02-07T12:22:33.450259+0000","last_undegraded":"2024-02-07T12:22:33.450259+0000","last_fullsized":"2024-02-07T12:22:33.450259+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:04:42.487324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451693+0000","last_change":"2024-02-07T12:22:05.431585+0000","last_active":"2024-02-07T12:22:33.451693+0000","last_peered":"2024-02-07T12:22:33.451693+0000","last_clean":"2024-02-07T12:22:33.451693+0000","last_became_active":"2024-02-07T12:22:05.431178+0000","last_became_peered":"2024-02-07T12:22:05.431178+0000","last_unstale":"2024-02-07T12:22:33.451693+0000","last_undegraded":"2024-02-07T12:22:33.451693+0000","last_fullsized":"2024-02-07T12:22:33.451693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:22:50.924026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.566163+0000","last_change":"2024-02-07T12:21:51.878602+0000","last_active":"2024-02-07T12:22:35.566163+0000","last_peered":"2024-02-07T12:22:35.566163+0000","last_clean":"2024-02-07T12:22:35.566163+0000","last_became_active":"2024-02-07T12:21:51.877629+0000","last_became_peered":"2024-02-07T12:21:51.877629+0000","last_unstale":"2024-02-07T12:22:35.566163+0000","last_undegraded":"2024-02-07T12:22:35.566163+0000","last_fullsized":"2024-02-07T12:22:35.566163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:21:50.856971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:21:50.856971+0000","last_clean_scrub_stamp":"2024-02-07T12:21:50.856971+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:56:58.825088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453141+0000","last_change":"2024-02-07T12:22:09.461339+0000","last_active":"2024-02-07T12:22:33.453141+0000","last_peered":"2024-02-07T12:22:33.453141+0000","last_clean":"2024-02-07T12:22:33.453141+0000","last_became_active":"2024-02-07T12:22:09.461004+0000","last_became_peered":"2024-02-07T12:22:09.461004+0000","last_unstale":"2024-02-07T12:22:33.453141+0000","last_undegraded":"2024-02-07T12:22:33.453141+0000","last_fullsized":"2024-02-07T12:22:33.453141+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:48:49.705656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451668+0000","last_change":"2024-02-07T12:22:11.481196+0000","last_active":"2024-02-07T12:22:33.451668+0000","last_peered":"2024-02-07T12:22:33.451668+0000","last_clean":"2024-02-07T12:22:33.451668+0000","last_became_active":"2024-02-07T12:22:11.480340+0000","last_became_peered":"2024-02-07T12:22:11.480340+0000","last_unstale":"2024-02-07T12:22:33.451668+0000","last_undegraded":"2024-02-07T12:22:33.451668+0000","last_fullsized":"2024-02-07T12:22:33.451668+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:34:57.392750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449968+0000","last_change":"2024-02-07T12:22:07.447297+0000","last_active":"2024-02-07T12:22:33.449968+0000","last_peered":"2024-02-07T12:22:33.449968+0000","last_clean":"2024-02-07T12:22:33.449968+0000","last_became_active":"2024-02-07T12:22:07.446966+0000","last_became_peered":"2024-02-07T12:22:07.446966+0000","last_unstale":"2024-02-07T12:22:33.449968+0000","last_undegraded":"2024-02-07T12:22:33.449968+0000","last_fullsized":"2024-02-07T12:22:33.449968+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:34:39.500538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452584+0000","last_change":"2024-02-07T12:22:05.436374+0000","last_active":"2024-02-07T12:22:33.452584+0000","last_peered":"2024-02-07T12:22:33.452584+0000","last_clean":"2024-02-07T12:22:33.452584+0000","last_became_active":"2024-02-07T12:22:05.436047+0000","last_became_peered":"2024-02-07T12:22:05.436047+0000","last_unstale":"2024-02-07T12:22:33.452584+0000","last_undegraded":"2024-02-07T12:22:33.452584+0000","last_fullsized":"2024-02-07T12:22:33.452584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:00:44.000402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455831+0000","last_change":"2024-02-07T12:22:09.465521+0000","last_active":"2024-02-07T12:22:33.455831+0000","last_peered":"2024-02-07T12:22:33.455831+0000","last_clean":"2024-02-07T12:22:33.455831+0000","last_became_active":"2024-02-07T12:22:09.464639+0000","last_became_peered":"2024-02-07T12:22:09.464639+0000","last_unstale":"2024-02-07T12:22:33.455831+0000","last_undegraded":"2024-02-07T12:22:33.455831+0000","last_fullsized":"2024-02-07T12:22:33.455831+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:21:01.449598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451068+0000","last_change":"2024-02-07T12:22:11.481507+0000","last_active":"2024-02-07T12:22:33.451068+0000","last_peered":"2024-02-07T12:22:33.451068+0000","last_clean":"2024-02-07T12:22:33.451068+0000","last_became_active":"2024-02-07T12:22:11.480608+0000","last_became_peered":"2024-02-07T12:22:11.480608+0000","last_unstale":"2024-02-07T12:22:33.451068+0000","last_undegraded":"2024-02-07T12:22:33.451068+0000","last_fullsized":"2024-02-07T12:22:33.451068+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:58:46.429737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.457057+0000","last_change":"2024-02-07T12:22:05.432607+0000","last_active":"2024-02-07T12:22:33.457057+0000","last_peered":"2024-02-07T12:22:33.457057+0000","last_clean":"2024-02-07T12:22:33.457057+0000","last_became_active":"2024-02-07T12:22:05.431011+0000","last_became_peered":"2024-02-07T12:22:05.431011+0000","last_unstale":"2024-02-07T12:22:33.457057+0000","last_undegraded":"2024-02-07T12:22:33.457057+0000","last_fullsized":"2024-02-07T12:22:33.457057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:54:54.832166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478273+0000","last_change":"2024-02-07T12:22:07.444110+0000","last_active":"2024-02-07T12:22:35.478273+0000","last_peered":"2024-02-07T12:22:35.478273+0000","last_clean":"2024-02-07T12:22:35.478273+0000","last_became_active":"2024-02-07T12:22:07.443639+0000","last_became_peered":"2024-02-07T12:22:07.443639+0000","last_unstale":"2024-02-07T12:22:35.478273+0000","last_undegraded":"2024-02-07T12:22:35.478273+0000","last_fullsized":"2024-02-07T12:22:35.478273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:13:24.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478426+0000","last_change":"2024-02-07T12:22:09.465667+0000","last_active":"2024-02-07T12:22:35.478426+0000","last_peered":"2024-02-07T12:22:35.478426+0000","last_clean":"2024-02-07T12:22:35.478426+0000","last_became_active":"2024-02-07T12:22:09.464933+0000","last_became_peered":"2024-02-07T12:22:09.464933+0000","last_unstale":"2024-02-07T12:22:35.478426+0000","last_undegraded":"2024-02-07T12:22:35.478426+0000","last_fullsized":"2024-02-07T12:22:35.478426+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:46:51.835370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450285+0000","last_change":"2024-02-07T12:22:11.482271+0000","last_active":"2024-02-07T12:22:33.450285+0000","last_peered":"2024-02-07T12:22:33.450285+0000","last_clean":"2024-02-07T12:22:33.450285+0000","last_became_active":"2024-02-07T12:22:11.481514+0000","last_became_peered":"2024-02-07T12:22:11.481514+0000","last_unstale":"2024-02-07T12:22:33.450285+0000","last_undegraded":"2024-02-07T12:22:33.450285+0000","last_fullsized":"2024-02-07T12:22:33.450285+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:32:36.638540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452574+0000","last_change":"2024-02-07T12:22:05.436878+0000","last_active":"2024-02-07T12:22:33.452574+0000","last_peered":"2024-02-07T12:22:33.452574+0000","last_clean":"2024-02-07T12:22:33.452574+0000","last_became_active":"2024-02-07T12:22:05.436605+0000","last_became_peered":"2024-02-07T12:22:05.436605+0000","last_unstale":"2024-02-07T12:22:33.452574+0000","last_undegraded":"2024-02-07T12:22:33.452574+0000","last_fullsized":"2024-02-07T12:22:33.452574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:34:00.686024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453753+0000","last_change":"2024-02-07T12:22:07.451080+0000","last_active":"2024-02-07T12:22:33.453753+0000","last_peered":"2024-02-07T12:22:33.453753+0000","last_clean":"2024-02-07T12:22:33.453753+0000","last_became_active":"2024-02-07T12:22:07.450700+0000","last_became_peered":"2024-02-07T12:22:07.450700+0000","last_unstale":"2024-02-07T12:22:33.453753+0000","last_undegraded":"2024-02-07T12:22:33.453753+0000","last_fullsized":"2024-02-07T12:22:33.453753+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:11:00.332249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478265+0000","last_change":"2024-02-07T12:22:09.467882+0000","last_active":"2024-02-07T12:22:35.478265+0000","last_peered":"2024-02-07T12:22:35.478265+0000","last_clean":"2024-02-07T12:22:35.478265+0000","last_became_active":"2024-02-07T12:22:09.467307+0000","last_became_peered":"2024-02-07T12:22:09.467307+0000","last_unstale":"2024-02-07T12:22:35.478265+0000","last_undegraded":"2024-02-07T12:22:35.478265+0000","last_fullsized":"2024-02-07T12:22:35.478265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:41:49.322998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455223+0000","last_change":"2024-02-07T12:22:11.476130+0000","last_active":"2024-02-07T12:22:33.455223+0000","last_peered":"2024-02-07T12:22:33.455223+0000","last_clean":"2024-02-07T12:22:33.455223+0000","last_became_active":"2024-02-07T12:22:11.475838+0000","last_became_peered":"2024-02-07T12:22:11.475838+0000","last_unstale":"2024-02-07T12:22:33.455223+0000","last_undegraded":"2024-02-07T12:22:33.455223+0000","last_fullsized":"2024-02-07T12:22:33.455223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:48:12.085334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453004+0000","last_change":"2024-02-07T12:22:05.435540+0000","last_active":"2024-02-07T12:22:33.453004+0000","last_peered":"2024-02-07T12:22:33.453004+0000","last_clean":"2024-02-07T12:22:33.453004+0000","last_became_active":"2024-02-07T12:22:05.435182+0000","last_became_peered":"2024-02-07T12:22:05.435182+0000","last_unstale":"2024-02-07T12:22:33.453004+0000","last_undegraded":"2024-02-07T12:22:33.453004+0000","last_fullsized":"2024-02-07T12:22:33.453004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:09:58.516333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452888+0000","last_change":"2024-02-07T12:22:07.451226+0000","last_active":"2024-02-07T12:22:33.452888+0000","last_peered":"2024-02-07T12:22:33.452888+0000","last_clean":"2024-02-07T12:22:33.452888+0000","last_became_active":"2024-02-07T12:22:07.450828+0000","last_became_peered":"2024-02-07T12:22:07.450828+0000","last_unstale":"2024-02-07T12:22:33.452888+0000","last_undegraded":"2024-02-07T12:22:33.452888+0000","last_fullsized":"2024-02-07T12:22:33.452888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:45:47.115738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454658+0000","last_change":"2024-02-07T12:22:09.464609+0000","last_active":"2024-02-07T12:22:33.454658+0000","last_peered":"2024-02-07T12:22:33.454658+0000","last_clean":"2024-02-07T12:22:33.454658+0000","last_became_active":"2024-02-07T12:22:09.463612+0000","last_became_peered":"2024-02-07T12:22:09.463612+0000","last_unstale":"2024-02-07T12:22:33.454658+0000","last_undegraded":"2024-02-07T12:22:33.454658+0000","last_fullsized":"2024-02-07T12:22:33.454658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:49:10.287793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452813+0000","last_change":"2024-02-07T12:22:11.480663+0000","last_active":"2024-02-07T12:22:33.452813+0000","last_peered":"2024-02-07T12:22:33.452813+0000","last_clean":"2024-02-07T12:22:33.452813+0000","last_became_active":"2024-02-07T12:22:11.480316+0000","last_became_peered":"2024-02-07T12:22:11.480316+0000","last_unstale":"2024-02-07T12:22:33.452813+0000","last_undegraded":"2024-02-07T12:22:33.452813+0000","last_fullsized":"2024-02-07T12:22:33.452813+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:12:22.960223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478406+0000","last_change":"2024-02-07T12:22:09.463477+0000","last_active":"2024-02-07T12:22:35.478406+0000","last_peered":"2024-02-07T12:22:35.478406+0000","last_clean":"2024-02-07T12:22:35.478406+0000","last_became_active":"2024-02-07T12:22:09.463165+0000","last_became_peered":"2024-02-07T12:22:09.463165+0000","last_unstale":"2024-02-07T12:22:35.478406+0000","last_undegraded":"2024-02-07T12:22:35.478406+0000","last_fullsized":"2024-02-07T12:22:35.478406+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:30:24.481576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450337+0000","last_change":"2024-02-07T12:22:05.436478+0000","last_active":"2024-02-07T12:22:33.450337+0000","last_peered":"2024-02-07T12:22:33.450337+0000","last_clean":"2024-02-07T12:22:33.450337+0000","last_became_active":"2024-02-07T12:22:05.436198+0000","last_became_peered":"2024-02-07T12:22:05.436198+0000","last_unstale":"2024-02-07T12:22:33.450337+0000","last_undegraded":"2024-02-07T12:22:33.450337+0000","last_fullsized":"2024-02-07T12:22:33.450337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:36:44.041796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455141+0000","last_change":"2024-02-07T12:22:07.449899+0000","last_active":"2024-02-07T12:22:33.455141+0000","last_peered":"2024-02-07T12:22:33.455141+0000","last_clean":"2024-02-07T12:22:33.455141+0000","last_became_active":"2024-02-07T12:22:07.449675+0000","last_became_peered":"2024-02-07T12:22:07.449675+0000","last_unstale":"2024-02-07T12:22:33.455141+0000","last_undegraded":"2024-02-07T12:22:33.455141+0000","last_fullsized":"2024-02-07T12:22:33.455141+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:15:17.745640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456918+0000","last_change":"2024-02-07T12:22:11.478981+0000","last_active":"2024-02-07T12:22:33.456918+0000","last_peered":"2024-02-07T12:22:33.456918+0000","last_clean":"2024-02-07T12:22:33.456918+0000","last_became_active":"2024-02-07T12:22:11.478681+0000","last_became_peered":"2024-02-07T12:22:11.478681+0000","last_unstale":"2024-02-07T12:22:33.456918+0000","last_undegraded":"2024-02-07T12:22:33.456918+0000","last_fullsized":"2024-02-07T12:22:33.456918+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:18:24.259519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453390+0000","last_change":"2024-02-07T12:22:09.466184+0000","last_active":"2024-02-07T12:22:33.453390+0000","last_peered":"2024-02-07T12:22:33.453390+0000","last_clean":"2024-02-07T12:22:33.453390+0000","last_became_active":"2024-02-07T12:22:09.465747+0000","last_became_peered":"2024-02-07T12:22:09.465747+0000","last_unstale":"2024-02-07T12:22:33.453390+0000","last_undegraded":"2024-02-07T12:22:33.453390+0000","last_fullsized":"2024-02-07T12:22:33.453390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:55:00.286671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450242+0000","last_change":"2024-02-07T12:22:05.437136+0000","last_active":"2024-02-07T12:22:33.450242+0000","last_peered":"2024-02-07T12:22:33.450242+0000","last_clean":"2024-02-07T12:22:33.450242+0000","last_became_active":"2024-02-07T12:22:05.436856+0000","last_became_peered":"2024-02-07T12:22:05.436856+0000","last_unstale":"2024-02-07T12:22:33.450242+0000","last_undegraded":"2024-02-07T12:22:33.450242+0000","last_fullsized":"2024-02-07T12:22:33.450242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:37:36.067961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454962+0000","last_change":"2024-02-07T12:22:07.450981+0000","last_active":"2024-02-07T12:22:33.454962+0000","last_peered":"2024-02-07T12:22:33.454962+0000","last_clean":"2024-02-07T12:22:33.454962+0000","last_became_active":"2024-02-07T12:22:07.450728+0000","last_became_peered":"2024-02-07T12:22:07.450728+0000","last_unstale":"2024-02-07T12:22:33.454962+0000","last_undegraded":"2024-02-07T12:22:33.454962+0000","last_fullsized":"2024-02-07T12:22:33.454962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:11:21.727261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455749+0000","last_change":"2024-02-07T12:22:11.478740+0000","last_active":"2024-02-07T12:22:33.455749+0000","last_peered":"2024-02-07T12:22:33.455749+0000","last_clean":"2024-02-07T12:22:33.455749+0000","last_became_active":"2024-02-07T12:22:11.478446+0000","last_became_peered":"2024-02-07T12:22:11.478446+0000","last_unstale":"2024-02-07T12:22:33.455749+0000","last_undegraded":"2024-02-07T12:22:33.455749+0000","last_fullsized":"2024-02-07T12:22:33.455749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:52:17.097751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456010+0000","last_change":"2024-02-07T12:22:05.432895+0000","last_active":"2024-02-07T12:22:33.456010+0000","last_peered":"2024-02-07T12:22:33.456010+0000","last_clean":"2024-02-07T12:22:33.456010+0000","last_became_active":"2024-02-07T12:22:05.431968+0000","last_became_peered":"2024-02-07T12:22:05.431968+0000","last_unstale":"2024-02-07T12:22:33.456010+0000","last_undegraded":"2024-02-07T12:22:33.456010+0000","last_fullsized":"2024-02-07T12:22:33.456010+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:49:33.225778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451597+0000","last_change":"2024-02-07T12:22:07.443455+0000","last_active":"2024-02-07T12:22:33.451597+0000","last_peered":"2024-02-07T12:22:33.451597+0000","last_clean":"2024-02-07T12:22:33.451597+0000","last_became_active":"2024-02-07T12:22:07.443079+0000","last_became_peered":"2024-02-07T12:22:07.443079+0000","last_unstale":"2024-02-07T12:22:33.451597+0000","last_undegraded":"2024-02-07T12:22:33.451597+0000","last_fullsized":"2024-02-07T12:22:33.451597+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:12:14.809707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451233+0000","last_change":"2024-02-07T12:22:09.461182+0000","last_active":"2024-02-07T12:22:33.451233+0000","last_peered":"2024-02-07T12:22:33.451233+0000","last_clean":"2024-02-07T12:22:33.451233+0000","last_became_active":"2024-02-07T12:22:09.460804+0000","last_became_peered":"2024-02-07T12:22:09.460804+0000","last_unstale":"2024-02-07T12:22:33.451233+0000","last_undegraded":"2024-02-07T12:22:33.451233+0000","last_fullsized":"2024-02-07T12:22:33.451233+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:44:49.012329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451588+0000","last_change":"2024-02-07T12:22:11.481792+0000","last_active":"2024-02-07T12:22:33.451588+0000","last_peered":"2024-02-07T12:22:33.451588+0000","last_clean":"2024-02-07T12:22:33.451588+0000","last_became_active":"2024-02-07T12:22:11.481457+0000","last_became_peered":"2024-02-07T12:22:11.481457+0000","last_unstale":"2024-02-07T12:22:33.451588+0000","last_undegraded":"2024-02-07T12:22:33.451588+0000","last_fullsized":"2024-02-07T12:22:33.451588+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:07:36.450089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.478468+0000","last_change":"2024-02-07T12:22:09.463439+0000","last_active":"2024-02-07T12:22:35.478468+0000","last_peered":"2024-02-07T12:22:35.478468+0000","last_clean":"2024-02-07T12:22:35.478468+0000","last_became_active":"2024-02-07T12:22:09.463107+0000","last_became_peered":"2024-02-07T12:22:09.463107+0000","last_unstale":"2024-02-07T12:22:35.478468+0000","last_undegraded":"2024-02-07T12:22:35.478468+0000","last_fullsized":"2024-02-07T12:22:35.478468+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:36:40.943193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456761+0000","last_change":"2024-02-07T12:22:05.434900+0000","last_active":"2024-02-07T12:22:33.456761+0000","last_peered":"2024-02-07T12:22:33.456761+0000","last_clean":"2024-02-07T12:22:33.456761+0000","last_became_active":"2024-02-07T12:22:05.432480+0000","last_became_peered":"2024-02-07T12:22:05.432480+0000","last_unstale":"2024-02-07T12:22:33.456761+0000","last_undegraded":"2024-02-07T12:22:33.456761+0000","last_fullsized":"2024-02-07T12:22:33.456761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:52:58.113664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450908+0000","last_change":"2024-02-07T12:22:07.445952+0000","last_active":"2024-02-07T12:22:33.450908+0000","last_peered":"2024-02-07T12:22:33.450908+0000","last_clean":"2024-02-07T12:22:33.450908+0000","last_became_active":"2024-02-07T12:22:07.445656+0000","last_became_peered":"2024-02-07T12:22:07.445656+0000","last_unstale":"2024-02-07T12:22:33.450908+0000","last_undegraded":"2024-02-07T12:22:33.450908+0000","last_fullsized":"2024-02-07T12:22:33.450908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:09:09.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455120+0000","last_change":"2024-02-07T12:22:11.476031+0000","last_active":"2024-02-07T12:22:33.455120+0000","last_peered":"2024-02-07T12:22:33.455120+0000","last_clean":"2024-02-07T12:22:33.455120+0000","last_became_active":"2024-02-07T12:22:11.475699+0000","last_became_peered":"2024-02-07T12:22:11.475699+0000","last_unstale":"2024-02-07T12:22:33.455120+0000","last_undegraded":"2024-02-07T12:22:33.455120+0000","last_fullsized":"2024-02-07T12:22:33.455120+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:26:07.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455475+0000","last_change":"2024-02-07T12:22:05.430181+0000","last_active":"2024-02-07T12:22:33.455475+0000","last_peered":"2024-02-07T12:22:33.455475+0000","last_clean":"2024-02-07T12:22:33.455475+0000","last_became_active":"2024-02-07T12:22:05.429629+0000","last_became_peered":"2024-02-07T12:22:05.429629+0000","last_unstale":"2024-02-07T12:22:33.455475+0000","last_undegraded":"2024-02-07T12:22:33.455475+0000","last_fullsized":"2024-02-07T12:22:33.455475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:38:11.319198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453308+0000","last_change":"2024-02-07T12:22:07.442136+0000","last_active":"2024-02-07T12:22:33.453308+0000","last_peered":"2024-02-07T12:22:33.453308+0000","last_clean":"2024-02-07T12:22:33.453308+0000","last_became_active":"2024-02-07T12:22:07.441582+0000","last_became_peered":"2024-02-07T12:22:07.441582+0000","last_unstale":"2024-02-07T12:22:33.453308+0000","last_undegraded":"2024-02-07T12:22:33.453308+0000","last_fullsized":"2024-02-07T12:22:33.453308+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:50:34.243579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450722+0000","last_change":"2024-02-07T12:22:09.463092+0000","last_active":"2024-02-07T12:22:33.450722+0000","last_peered":"2024-02-07T12:22:33.450722+0000","last_clean":"2024-02-07T12:22:33.450722+0000","last_became_active":"2024-02-07T12:22:09.462724+0000","last_became_peered":"2024-02-07T12:22:09.462724+0000","last_unstale":"2024-02-07T12:22:33.450722+0000","last_undegraded":"2024-02-07T12:22:33.450722+0000","last_fullsized":"2024-02-07T12:22:33.450722+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:08:56.226751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449758+0000","last_change":"2024-02-07T12:22:11.482625+0000","last_active":"2024-02-07T12:22:33.449758+0000","last_peered":"2024-02-07T12:22:33.449758+0000","last_clean":"2024-02-07T12:22:33.449758+0000","last_became_active":"2024-02-07T12:22:11.482329+0000","last_became_peered":"2024-02-07T12:22:11.482329+0000","last_unstale":"2024-02-07T12:22:33.449758+0000","last_undegraded":"2024-02-07T12:22:33.449758+0000","last_fullsized":"2024-02-07T12:22:33.449758+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:20:04.028005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456118+0000","last_change":"2024-02-07T12:22:05.430903+0000","last_active":"2024-02-07T12:22:33.456118+0000","last_peered":"2024-02-07T12:22:33.456118+0000","last_clean":"2024-02-07T12:22:33.456118+0000","last_became_active":"2024-02-07T12:22:05.430618+0000","last_became_peered":"2024-02-07T12:22:05.430618+0000","last_unstale":"2024-02-07T12:22:33.456118+0000","last_undegraded":"2024-02-07T12:22:33.456118+0000","last_fullsized":"2024-02-07T12:22:33.456118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:53:58.556906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451413+0000","last_change":"2024-02-07T12:22:07.447445+0000","last_active":"2024-02-07T12:22:33.451413+0000","last_peered":"2024-02-07T12:22:33.451413+0000","last_clean":"2024-02-07T12:22:33.451413+0000","last_became_active":"2024-02-07T12:22:07.445865+0000","last_became_peered":"2024-02-07T12:22:07.445865+0000","last_unstale":"2024-02-07T12:22:33.451413+0000","last_undegraded":"2024-02-07T12:22:33.451413+0000","last_fullsized":"2024-02-07T12:22:33.451413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:57:00.770216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451186+0000","last_change":"2024-02-07T12:22:09.465013+0000","last_active":"2024-02-07T12:22:33.451186+0000","last_peered":"2024-02-07T12:22:33.451186+0000","last_clean":"2024-02-07T12:22:33.451186+0000","last_became_active":"2024-02-07T12:22:09.464676+0000","last_became_peered":"2024-02-07T12:22:09.464676+0000","last_unstale":"2024-02-07T12:22:33.451186+0000","last_undegraded":"2024-02-07T12:22:33.451186+0000","last_fullsized":"2024-02-07T12:22:33.451186+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:35:34.706682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450470+0000","last_change":"2024-02-07T12:22:11.482238+0000","last_active":"2024-02-07T12:22:33.450470+0000","last_peered":"2024-02-07T12:22:33.450470+0000","last_clean":"2024-02-07T12:22:33.450470+0000","last_became_active":"2024-02-07T12:22:11.480961+0000","last_became_peered":"2024-02-07T12:22:11.480961+0000","last_unstale":"2024-02-07T12:22:33.450470+0000","last_undegraded":"2024-02-07T12:22:33.450470+0000","last_fullsized":"2024-02-07T12:22:33.450470+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:00:17.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452795+0000","last_change":"2024-02-07T12:22:05.433668+0000","last_active":"2024-02-07T12:22:33.452795+0000","last_peered":"2024-02-07T12:22:33.452795+0000","last_clean":"2024-02-07T12:22:33.452795+0000","last_became_active":"2024-02-07T12:22:05.433406+0000","last_became_peered":"2024-02-07T12:22:05.433406+0000","last_unstale":"2024-02-07T12:22:33.452795+0000","last_undegraded":"2024-02-07T12:22:33.452795+0000","last_fullsized":"2024-02-07T12:22:33.452795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:22:41.076847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453097+0000","last_change":"2024-02-07T12:22:07.442138+0000","last_active":"2024-02-07T12:22:33.453097+0000","last_peered":"2024-02-07T12:22:33.453097+0000","last_clean":"2024-02-07T12:22:33.453097+0000","last_became_active":"2024-02-07T12:22:07.441874+0000","last_became_peered":"2024-02-07T12:22:07.441874+0000","last_unstale":"2024-02-07T12:22:33.453097+0000","last_undegraded":"2024-02-07T12:22:33.453097+0000","last_fullsized":"2024-02-07T12:22:33.453097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:29:13.828193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455128+0000","last_change":"2024-02-07T12:22:09.465552+0000","last_active":"2024-02-07T12:22:33.455128+0000","last_peered":"2024-02-07T12:22:33.455128+0000","last_clean":"2024-02-07T12:22:33.455128+0000","last_became_active":"2024-02-07T12:22:09.464794+0000","last_became_peered":"2024-02-07T12:22:09.464794+0000","last_unstale":"2024-02-07T12:22:33.455128+0000","last_undegraded":"2024-02-07T12:22:33.455128+0000","last_fullsized":"2024-02-07T12:22:33.455128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:04:40.717522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738391,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 7 12:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 7 12:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-07T12:22:48.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-07T12:22:48.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:48 smithi171 ceph-mon[30005]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:22:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:48.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:48 smithi106 ceph-mon[29693]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:22:48.787 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:48 smithi078 ceph-mon[25576]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:22:49.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:49 smithi078 ceph-mon[25576]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T12:22:49.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:49 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:22:49.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:49 smithi171 ceph-mon[30005]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T12:22:49.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:49 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:22:49.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:49 smithi106 ceph-mon[29693]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T12:22:49.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:49 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:22:51.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:51 smithi078 ceph-mon[25576]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T12:22:51.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:51 smithi171 ceph-mon[30005]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T12:22:51.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:51 smithi106 ceph-mon[29693]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T12:22:52.026 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:22:52.028 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-02-07T12:22:52.325 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:52 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ::ffff:172.21.15.106 - - [07/Feb/2024:12:22:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T12:22:53.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:53 smithi078 ceph-mon[25576]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:53.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:53 smithi078 ceph-mon[25576]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:22:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-02-07T12:22:51.464708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96612,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126556,"statfs":{"total":287972524032,"available":287873593344,"internally_reserved":0,"allocated":3158016,"data_stored":2100177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4783,"internal_metadata":95743313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002884"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451169+0000","last_change":"2024-02-07T12:22:11.481050+0000","last_active":"2024-02-07T12:22:33.451169+0000","last_peered":"2024-02-07T12:22:33.451169+0000","last_clean":"2024-02-07T12:22:33.451169+0000","last_became_active":"2024-02-07T12:22:11.480384+0000","last_became_peered":"2024-02-07T12:22:11.480384+0000","last_unstale":"2024-02-07T12:22:33.451169+0000","last_undegraded":"2024-02-07T12:22:33.451169+0000","last_fullsized":"2024-02-07T12:22:33.451169+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:51:48.869912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454450+0000","last_change":"2024-02-07T12:22:09.463584+0000","last_active":"2024-02-07T12:22:33.454450+0000","last_peered":"2024-02-07T12:22:33.454450+0000","last_clean":"2024-02-07T12:22:33.454450+0000","last_became_active":"2024-02-07T12:22:09.462997+0000","last_became_peered":"2024-02-07T12:22:09.462997+0000","last_unstale":"2024-02-07T12:22:33.454450+0000","last_undegraded":"2024-02-07T12:22:33.454450+0000","last_fullsized":"2024-02-07T12:22:33.454450+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:40:58.176999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453676+0000","last_change":"2024-02-07T12:22:05.437143+0000","last_active":"2024-02-07T12:22:33.453676+0000","last_peered":"2024-02-07T12:22:33.453676+0000","last_clean":"2024-02-07T12:22:33.453676+0000","last_became_active":"2024-02-07T12:22:05.436561+0000","last_became_peered":"2024-02-07T12:22:05.436561+0000","last_unstale":"2024-02-07T12:22:33.453676+0000","last_undegraded":"2024-02-07T12:22:33.453676+0000","last_fullsized":"2024-02-07T12:22:33.453676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:51:44.462007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452797+0000","last_change":"2024-02-07T12:22:07.449909+0000","last_active":"2024-02-07T12:22:33.452797+0000","last_peered":"2024-02-07T12:22:33.452797+0000","last_clean":"2024-02-07T12:22:33.452797+0000","last_became_active":"2024-02-07T12:22:07.449549+0000","last_became_peered":"2024-02-07T12:22:07.449549+0000","last_unstale":"2024-02-07T12:22:33.452797+0000","last_undegraded":"2024-02-07T12:22:33.452797+0000","last_fullsized":"2024-02-07T12:22:33.452797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:55:40.361392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450157+0000","last_change":"2024-02-07T12:22:11.485021+0000","last_active":"2024-02-07T12:22:33.450157+0000","last_peered":"2024-02-07T12:22:33.450157+0000","last_clean":"2024-02-07T12:22:33.450157+0000","last_became_active":"2024-02-07T12:22:11.484690+0000","last_became_peered":"2024-02-07T12:22:11.484690+0000","last_unstale":"2024-02-07T12:22:33.450157+0000","last_undegraded":"2024-02-07T12:22:33.450157+0000","last_fullsized":"2024-02-07T12:22:33.450157+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:33:06.201071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451022+0000","last_change":"2024-02-07T12:22:09.462919+0000","last_active":"2024-02-07T12:22:33.451022+0000","last_peered":"2024-02-07T12:22:33.451022+0000","last_clean":"2024-02-07T12:22:33.451022+0000","last_became_active":"2024-02-07T12:22:09.462545+0000","last_became_peered":"2024-02-07T12:22:09.462545+0000","last_unstale":"2024-02-07T12:22:33.451022+0000","last_undegraded":"2024-02-07T12:22:33.451022+0000","last_fullsized":"2024-02-07T12:22:33.451022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:38:59.919191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453009+0000","last_change":"2024-02-07T12:22:05.437695+0000","last_active":"2024-02-07T12:22:33.453009+0000","last_peered":"2024-02-07T12:22:33.453009+0000","last_clean":"2024-02-07T12:22:33.453009+0000","last_became_active":"2024-02-07T12:22:05.437395+0000","last_became_peered":"2024-02-07T12:22:05.437395+0000","last_unstale":"2024-02-07T12:22:33.453009+0000","last_undegraded":"2024-02-07T12:22:33.453009+0000","last_fullsized":"2024-02-07T12:22:33.453009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:30:28.867141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453452+0000","last_change":"2024-02-07T12:22:07.443515+0000","last_active":"2024-02-07T12:22:33.453452+0000","last_peered":"2024-02-07T12:22:33.453452+0000","last_clean":"2024-02-07T12:22:33.453452+0000","last_became_active":"2024-02-07T12:22:07.443037+0000","last_became_peered":"2024-02-07T12:22:07.443037+0000","last_unstale":"2024-02-07T12:22:33.453452+0000","last_undegraded":"2024-02-07T12:22:33.453452+0000","last_fullsized":"2024-02-07T12:22:33.453452+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:07:36.609536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455367+0000","last_change":"2024-02-07T12:22:11.475782+0000","last_active":"2024-02-07T12:22:33.455367+0000","last_peered":"2024-02-07T12:22:33.455367+0000","last_clean":"2024-02-07T12:22:33.455367+0000","last_became_active":"2024-02-07T12:22:11.475487+0000","last_became_peered":"2024-02-07T12:22:11.475487+0000","last_unstale":"2024-02-07T12:22:33.455367+0000","last_undegraded":"2024-02-07T12:22:33.455367+0000","last_fullsized":"2024-02-07T12:22:33.455367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:34:11.990286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451738+0000","last_change":"2024-02-07T12:22:09.463619+0000","last_active":"2024-02-07T12:22:33.451738+0000","last_peered":"2024-02-07T12:22:33.451738+0000","last_clean":"2024-02-07T12:22:33.451738+0000","last_became_active":"2024-02-07T12:22:09.463245+0000","last_became_peered":"2024-02-07T12:22:09.463245+0000","last_unstale":"2024-02-07T12:22:33.451738+0000","last_undegraded":"2024-02-07T12:22:33.451738+0000","last_fullsized":"2024-02-07T12:22:33.451738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:57:37.634746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.457211+0000","last_change":"2024-02-07T12:22:05.434723+0000","last_active":"2024-02-07T12:22:33.457211+0000","last_peered":"2024-02-07T12:22:33.457211+0000","last_clean":"2024-02-07T12:22:33.457211+0000","last_became_active":"2024-02-07T12:22:05.431841+0000","last_became_peered":"2024-02-07T12:22:05.431841+0000","last_unstale":"2024-02-07T12:22:33.457211+0000","last_undegraded":"2024-02-07T12:22:33.457211+0000","last_fullsized":"2024-02-07T12:22:33.457211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:33:56.981715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451466+0000","last_change":"2024-02-07T12:22:07.448254+0000","last_active":"2024-02-07T12:22:33.451466+0000","last_peered":"2024-02-07T12:22:33.451466+0000","last_clean":"2024-02-07T12:22:33.451466+0000","last_became_active":"2024-02-07T12:22:07.447711+0000","last_became_peered":"2024-02-07T12:22:07.447711+0000","last_unstale":"2024-02-07T12:22:33.451466+0000","last_undegraded":"2024-02-07T12:22:33.451466+0000","last_fullsized":"2024-02-07T12:22:33.451466+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:10:28.271712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450782+0000","last_change":"2024-02-07T12:22:11.483371+0000","last_active":"2024-02-07T12:22:33.450782+0000","last_peered":"2024-02-07T12:22:33.450782+0000","last_clean":"2024-02-07T12:22:33.450782+0000","last_became_active":"2024-02-07T12:22:11.481464+0000","last_became_peered":"2024-02-07T12:22:11.481464+0000","last_unstale":"2024-02-07T12:22:33.450782+0000","last_undegraded":"2024-02-07T12:22:33.450782+0000","last_fullsized":"2024-02-07T12:22:33.450782+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:42:16.657815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451163+0000","last_change":"2024-02-07T12:22:09.462401+0000","last_active":"2024-02-07T12:22:33.451163+0000","last_peered":"2024-02-07T12:22:33.451163+0000","last_clean":"2024-02-07T12:22:33.451163+0000","last_became_active":"2024-02-07T12:22:09.462083+0000","last_became_peered":"2024-02-07T12:22:09.462083+0000","last_unstale":"2024-02-07T12:22:33.451163+0000","last_undegraded":"2024-02-07T12:22:33.451163+0000","last_fullsized":"2024-02-07T12:22:33.451163+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:17:52.651864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455414+0000","last_change":"2024-02-07T12:22:05.432478+0000","last_active":"2024-02-07T12:22:33.455414+0000","last_peered":"2024-02-07T12:22:33.455414+0000","last_clean":"2024-02-07T12:22:33.455414+0000","last_became_active":"2024-02-07T12:22:05.431921+0000","last_became_peered":"2024-02-07T12:22:05.431921+0000","last_unstale":"2024-02-07T12:22:33.455414+0000","last_undegraded":"2024-02-07T12:22:33.455414+0000","last_fullsized":"2024-02-07T12:22:33.455414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:48:08.676509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.457156+0000","last_change":"2024-02-07T12:22:07.450762+0000","last_active":"2024-02-07T12:22:33.457156+0000","last_peered":"2024-02-07T12:22:33.457156+0000","last_clean":"2024-02-07T12:22:33.457156+0000","last_became_active":"2024-02-07T12:22:07.450420+0000","last_became_peered":"2024-02-07T12:22:07.450420+0000","last_unstale":"2024-02-07T12:22:33.457156+0000","last_undegraded":"2024-02-07T12:22:33.457156+0000","last_fullsized":"2024-02-07T12:22:33.457156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:59:46.913171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451191+0000","last_change":"2024-02-07T12:22:11.482105+0000","last_active":"2024-02-07T12:22:33.451191+0000","last_peered":"2024-02-07T12:22:33.451191+0000","last_clean":"2024-02-07T12:22:33.451191+0000","last_became_active":"2024-02-07T12:22:11.481719+0000","last_became_peered":"2024-02-07T12:22:11.481719+0000","last_unstale":"2024-02-07T12:22:33.451191+0000","last_undegraded":"2024-02-07T12:22:33.451191+0000","last_fullsized":"2024-02-07T12:22:33.451191+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:02:37.188006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455844+0000","last_change":"2024-02-07T12:22:09.464830+0000","last_active":"2024-02-07T12:22:33.455844+0000","last_peered":"2024-02-07T12:22:33.455844+0000","last_clean":"2024-02-07T12:22:33.455844+0000","last_became_active":"2024-02-07T12:22:09.463802+0000","last_became_peered":"2024-02-07T12:22:09.463802+0000","last_unstale":"2024-02-07T12:22:33.455844+0000","last_undegraded":"2024-02-07T12:22:33.455844+0000","last_fullsized":"2024-02-07T12:22:33.455844+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:32:17.465677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456850+0000","last_change":"2024-02-07T12:22:05.434627+0000","last_active":"2024-02-07T12:22:33.456850+0000","last_peered":"2024-02-07T12:22:33.456850+0000","last_clean":"2024-02-07T12:22:33.456850+0000","last_became_active":"2024-02-07T12:22:05.432882+0000","last_became_peered":"2024-02-07T12:22:05.432882+0000","last_unstale":"2024-02-07T12:22:33.456850+0000","last_undegraded":"2024-02-07T12:22:33.456850+0000","last_fullsized":"2024-02-07T12:22:33.456850+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:59:52.006199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456030+0000","last_change":"2024-02-07T12:22:07.449956+0000","last_active":"2024-02-07T12:22:33.456030+0000","last_peered":"2024-02-07T12:22:33.456030+0000","last_clean":"2024-02-07T12:22:33.456030+0000","last_became_active":"2024-02-07T12:22:07.449683+0000","last_became_peered":"2024-02-07T12:22:07.449683+0000","last_unstale":"2024-02-07T12:22:33.456030+0000","last_undegraded":"2024-02-07T12:22:33.456030+0000","last_fullsized":"2024-02-07T12:22:33.456030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:58:12.419754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450683+0000","last_change":"2024-02-07T12:22:11.484506+0000","last_active":"2024-02-07T12:22:33.450683+0000","last_peered":"2024-02-07T12:22:33.450683+0000","last_clean":"2024-02-07T12:22:33.450683+0000","last_became_active":"2024-02-07T12:22:11.484115+0000","last_became_peered":"2024-02-07T12:22:11.484115+0000","last_unstale":"2024-02-07T12:22:33.450683+0000","last_undegraded":"2024-02-07T12:22:33.450683+0000","last_fullsized":"2024-02-07T12:22:33.450683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:07:35.410175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455303+0000","last_change":"2024-02-07T12:22:09.464581+0000","last_active":"2024-02-07T12:22:33.455303+0000","last_peered":"2024-02-07T12:22:33.455303+0000","last_clean":"2024-02-07T12:22:33.455303+0000","last_became_active":"2024-02-07T12:22:09.463821+0000","last_became_peered":"2024-02-07T12:22:09.463821+0000","last_unstale":"2024-02-07T12:22:33.455303+0000","last_undegraded":"2024-02-07T12:22:33.455303+0000","last_fullsized":"2024-02-07T12:22:33.455303+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:19:14.492671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451343+0000","last_change":"2024-02-07T12:22:05.437329+0000","last_active":"2024-02-07T12:22:33.451343+0000","last_peered":"2024-02-07T12:22:33.451343+0000","last_clean":"2024-02-07T12:22:33.451343+0000","last_became_active":"2024-02-07T12:22:05.437024+0000","last_became_peered":"2024-02-07T12:22:05.437024+0000","last_unstale":"2024-02-07T12:22:33.451343+0000","last_undegraded":"2024-02-07T12:22:33.451343+0000","last_fullsized":"2024-02-07T12:22:33.451343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:12:39.909665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456138+0000","last_change":"2024-02-07T12:22:07.451035+0000","last_active":"2024-02-07T12:22:33.456138+0000","last_peered":"2024-02-07T12:22:33.456138+0000","last_clean":"2024-02-07T12:22:33.456138+0000","last_became_active":"2024-02-07T12:22:07.449672+0000","last_became_peered":"2024-02-07T12:22:07.449672+0000","last_unstale":"2024-02-07T12:22:33.456138+0000","last_undegraded":"2024-02-07T12:22:33.456138+0000","last_fullsized":"2024-02-07T12:22:33.456138+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:27:09.879939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452117+0000","last_change":"2024-02-07T12:22:11.484559+0000","last_active":"2024-02-07T12:22:33.452117+0000","last_peered":"2024-02-07T12:22:33.452117+0000","last_clean":"2024-02-07T12:22:33.452117+0000","last_became_active":"2024-02-07T12:22:11.483907+0000","last_became_peered":"2024-02-07T12:22:11.483907+0000","last_unstale":"2024-02-07T12:22:33.452117+0000","last_undegraded":"2024-02-07T12:22:33.452117+0000","last_fullsized":"2024-02-07T12:22:33.452117+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:59:06.096473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479587+0000","last_change":"2024-02-07T12:22:09.465726+0000","last_active":"2024-02-07T12:22:40.479587+0000","last_peered":"2024-02-07T12:22:40.479587+0000","last_clean":"2024-02-07T12:22:40.479587+0000","last_became_active":"2024-02-07T12:22:09.464438+0000","last_became_peered":"2024-02-07T12:22:09.464438+0000","last_unstale":"2024-02-07T12:22:40.479587+0000","last_undegraded":"2024-02-07T12:22:40.479587+0000","last_fullsized":"2024-02-07T12:22:40.479587+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:37:20.915814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450384+0000","last_change":"2024-02-07T12:22:05.437758+0000","last_active":"2024-02-07T12:22:33.450384+0000","last_peered":"2024-02-07T12:22:33.450384+0000","last_clean":"2024-02-07T12:22:33.450384+0000","last_became_active":"2024-02-07T12:22:05.437453+0000","last_became_peered":"2024-02-07T12:22:05.437453+0000","last_unstale":"2024-02-07T12:22:33.450384+0000","last_undegraded":"2024-02-07T12:22:33.450384+0000","last_fullsized":"2024-02-07T12:22:33.450384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:37:24.386646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449457+0000","last_change":"2024-02-07T12:22:07.444158+0000","last_active":"2024-02-07T12:22:33.449457+0000","last_peered":"2024-02-07T12:22:33.449457+0000","last_clean":"2024-02-07T12:22:33.449457+0000","last_became_active":"2024-02-07T12:22:07.443846+0000","last_became_peered":"2024-02-07T12:22:07.443846+0000","last_unstale":"2024-02-07T12:22:33.449457+0000","last_undegraded":"2024-02-07T12:22:33.449457+0000","last_fullsized":"2024-02-07T12:22:33.449457+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:57:16.306199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451571+0000","last_change":"2024-02-07T12:22:11.483239+0000","last_active":"2024-02-07T12:22:33.451571+0000","last_peered":"2024-02-07T12:22:33.451571+0000","last_clean":"2024-02-07T12:22:33.451571+0000","last_became_active":"2024-02-07T12:22:11.481813+0000","last_became_peered":"2024-02-07T12:22:11.481813+0000","last_unstale":"2024-02-07T12:22:33.451571+0000","last_undegraded":"2024-02-07T12:22:33.451571+0000","last_fullsized":"2024-02-07T12:22:33.451571+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:25:35.175522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452288+0000","last_change":"2024-02-07T12:22:09.466309+0000","last_active":"2024-02-07T12:22:33.452288+0000","last_peered":"2024-02-07T12:22:33.452288+0000","last_clean":"2024-02-07T12:22:33.452288+0000","last_became_active":"2024-02-07T12:22:09.465935+0000","last_became_peered":"2024-02-07T12:22:09.465935+0000","last_unstale":"2024-02-07T12:22:33.452288+0000","last_undegraded":"2024-02-07T12:22:33.452288+0000","last_fullsized":"2024-02-07T12:22:33.452288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:52:32.912644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454510+0000","last_change":"2024-02-07T12:22:05.434566+0000","last_active":"2024-02-07T12:22:33.454510+0000","last_peered":"2024-02-07T12:22:33.454510+0000","last_clean":"2024-02-07T12:22:33.454510+0000","last_became_active":"2024-02-07T12:22:05.432441+0000","last_became_peered":"2024-02-07T12:22:05.432441+0000","last_unstale":"2024-02-07T12:22:33.454510+0000","last_undegraded":"2024-02-07T12:22:33.454510+0000","last_fullsized":"2024-02-07T12:22:33.454510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:39:13.404359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449693+0000","last_change":"2024-02-07T12:22:07.444290+0000","last_active":"2024-02-07T12:22:33.449693+0000","last_peered":"2024-02-07T12:22:33.449693+0000","last_clean":"2024-02-07T12:22:33.449693+0000","last_became_active":"2024-02-07T12:22:07.443906+0000","last_became_peered":"2024-02-07T12:22:07.443906+0000","last_unstale":"2024-02-07T12:22:33.449693+0000","last_undegraded":"2024-02-07T12:22:33.449693+0000","last_fullsized":"2024-02-07T12:22:33.449693+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:28:57.762165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451451+0000","last_change":"2024-02-07T12:22:11.480678+0000","last_active":"2024-02-07T12:22:33.451451+0000","last_peered":"2024-02-07T12:22:33.451451+0000","last_clean":"2024-02-07T12:22:33.451451+0000","last_became_active":"2024-02-07T12:22:11.479936+0000","last_became_peered":"2024-02-07T12:22:11.479936+0000","last_unstale":"2024-02-07T12:22:33.451451+0000","last_undegraded":"2024-02-07T12:22:33.451451+0000","last_fullsized":"2024-02-07T12:22:33.451451+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:21:27.000037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449584+0000","last_change":"2024-02-07T12:22:09.464966+0000","last_active":"2024-02-07T12:22:33.449584+0000","last_peered":"2024-02-07T12:22:33.449584+0000","last_clean":"2024-02-07T12:22:33.449584+0000","last_became_active":"2024-02-07T12:22:09.464491+0000","last_became_peered":"2024-02-07T12:22:09.464491+0000","last_unstale":"2024-02-07T12:22:33.449584+0000","last_undegraded":"2024-02-07T12:22:33.449584+0000","last_fullsized":"2024-02-07T12:22:33.449584+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:48:10.264312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453212+0000","last_change":"2024-02-07T12:22:05.432383+0000","last_active":"2024-02-07T12:22:33.453212+0000","last_peered":"2024-02-07T12:22:33.453212+0000","last_clean":"2024-02-07T12:22:33.453212+0000","last_became_active":"2024-02-07T12:22:05.432040+0000","last_became_peered":"2024-02-07T12:22:05.432040+0000","last_unstale":"2024-02-07T12:22:33.453212+0000","last_undegraded":"2024-02-07T12:22:33.453212+0000","last_fullsized":"2024-02-07T12:22:33.453212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:48:34.693979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452737+0000","last_change":"2024-02-07T12:22:07.441536+0000","last_active":"2024-02-07T12:22:33.452737+0000","last_peered":"2024-02-07T12:22:33.452737+0000","last_clean":"2024-02-07T12:22:33.452737+0000","last_became_active":"2024-02-07T12:22:07.441247+0000","last_became_peered":"2024-02-07T12:22:07.441247+0000","last_unstale":"2024-02-07T12:22:33.452737+0000","last_undegraded":"2024-02-07T12:22:33.452737+0000","last_fullsized":"2024-02-07T12:22:33.452737+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:24:17.950707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450857+0000","last_change":"2024-02-07T12:22:11.481427+0000","last_active":"2024-02-07T12:22:33.450857+0000","last_peered":"2024-02-07T12:22:33.450857+0000","last_clean":"2024-02-07T12:22:33.450857+0000","last_became_active":"2024-02-07T12:22:11.479878+0000","last_became_peered":"2024-02-07T12:22:11.479878+0000","last_unstale":"2024-02-07T12:22:33.450857+0000","last_undegraded":"2024-02-07T12:22:33.450857+0000","last_fullsized":"2024-02-07T12:22:33.450857+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:03:02.219923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449834+0000","last_change":"2024-02-07T12:22:09.467738+0000","last_active":"2024-02-07T12:22:33.449834+0000","last_peered":"2024-02-07T12:22:33.449834+0000","last_clean":"2024-02-07T12:22:33.449834+0000","last_became_active":"2024-02-07T12:22:09.467436+0000","last_became_peered":"2024-02-07T12:22:09.467436+0000","last_unstale":"2024-02-07T12:22:33.449834+0000","last_undegraded":"2024-02-07T12:22:33.449834+0000","last_fullsized":"2024-02-07T12:22:33.449834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:15:36.041735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452581+0000","last_change":"2024-02-07T12:22:05.434184+0000","last_active":"2024-02-07T12:22:33.452581+0000","last_peered":"2024-02-07T12:22:33.452581+0000","last_clean":"2024-02-07T12:22:33.452581+0000","last_became_active":"2024-02-07T12:22:05.433739+0000","last_became_peered":"2024-02-07T12:22:05.433739+0000","last_unstale":"2024-02-07T12:22:33.452581+0000","last_undegraded":"2024-02-07T12:22:33.452581+0000","last_fullsized":"2024-02-07T12:22:33.452581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:31:11.589928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453574+0000","last_change":"2024-02-07T12:22:07.446985+0000","last_active":"2024-02-07T12:22:33.453574+0000","last_peered":"2024-02-07T12:22:33.453574+0000","last_clean":"2024-02-07T12:22:33.453574+0000","last_became_active":"2024-02-07T12:22:07.446529+0000","last_became_peered":"2024-02-07T12:22:07.446529+0000","last_unstale":"2024-02-07T12:22:33.453574+0000","last_undegraded":"2024-02-07T12:22:33.453574+0000","last_fullsized":"2024-02-07T12:22:33.453574+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:52:48.183777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455466+0000","last_change":"2024-02-07T12:22:11.478689+0000","last_active":"2024-02-07T12:22:33.455466+0000","last_peered":"2024-02-07T12:22:33.455466+0000","last_clean":"2024-02-07T12:22:33.455466+0000","last_became_active":"2024-02-07T12:22:11.478405+0000","last_became_peered":"2024-02-07T12:22:11.478405+0000","last_unstale":"2024-02-07T12:22:33.455466+0000","last_undegraded":"2024-02-07T12:22:33.455466+0000","last_fullsized":"2024-02-07T12:22:33.455466+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:28:16.251286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451336+0000","last_change":"2024-02-07T12:22:09.462699+0000","last_active":"2024-02-07T12:22:33.451336+0000","last_peered":"2024-02-07T12:22:33.451336+0000","last_clean":"2024-02-07T12:22:33.451336+0000","last_became_active":"2024-02-07T12:22:09.462164+0000","last_became_peered":"2024-02-07T12:22:09.462164+0000","last_unstale":"2024-02-07T12:22:33.451336+0000","last_undegraded":"2024-02-07T12:22:33.451336+0000","last_fullsized":"2024-02-07T12:22:33.451336+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:46:53.340951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.457110+0000","last_change":"2024-02-07T12:22:05.436026+0000","last_active":"2024-02-07T12:22:33.457110+0000","last_peered":"2024-02-07T12:22:33.457110+0000","last_clean":"2024-02-07T12:22:33.457110+0000","last_became_active":"2024-02-07T12:22:05.434573+0000","last_became_peered":"2024-02-07T12:22:05.434573+0000","last_unstale":"2024-02-07T12:22:33.457110+0000","last_undegraded":"2024-02-07T12:22:33.457110+0000","last_fullsized":"2024-02-07T12:22:33.457110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:52:17.391908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451619+0000","last_change":"2024-02-07T12:22:07.447201+0000","last_active":"2024-02-07T12:22:33.451619+0000","last_peered":"2024-02-07T12:22:33.451619+0000","last_clean":"2024-02-07T12:22:33.451619+0000","last_became_active":"2024-02-07T12:22:07.445745+0000","last_became_peered":"2024-02-07T12:22:07.445745+0000","last_unstale":"2024-02-07T12:22:33.451619+0000","last_undegraded":"2024-02-07T12:22:33.451619+0000","last_fullsized":"2024-02-07T12:22:33.451619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:40:13.014137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454837+0000","last_change":"2024-02-07T12:22:11.475394+0000","last_active":"2024-02-07T12:22:33.454837+0000","last_peered":"2024-02-07T12:22:33.454837+0000","last_clean":"2024-02-07T12:22:33.454837+0000","last_became_active":"2024-02-07T12:22:11.475099+0000","last_became_peered":"2024-02-07T12:22:11.475099+0000","last_unstale":"2024-02-07T12:22:33.454837+0000","last_undegraded":"2024-02-07T12:22:33.454837+0000","last_fullsized":"2024-02-07T12:22:33.454837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:11:42.390618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450116+0000","last_change":"2024-02-07T12:22:09.465490+0000","last_active":"2024-02-07T12:22:33.450116+0000","last_peered":"2024-02-07T12:22:33.450116+0000","last_clean":"2024-02-07T12:22:33.450116+0000","last_became_active":"2024-02-07T12:22:09.465079+0000","last_became_peered":"2024-02-07T12:22:09.465079+0000","last_unstale":"2024-02-07T12:22:33.450116+0000","last_undegraded":"2024-02-07T12:22:33.450116+0000","last_fullsized":"2024-02-07T12:22:33.450116+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:55:18.079376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451367+0000","last_change":"2024-02-07T12:22:05.435525+0000","last_active":"2024-02-07T12:22:33.451367+0000","last_peered":"2024-02-07T12:22:33.451367+0000","last_clean":"2024-02-07T12:22:33.451367+0000","last_became_active":"2024-02-07T12:22:05.435110+0000","last_became_peered":"2024-02-07T12:22:05.435110+0000","last_unstale":"2024-02-07T12:22:33.451367+0000","last_undegraded":"2024-02-07T12:22:33.451367+0000","last_fullsized":"2024-02-07T12:22:33.451367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:43:03.793348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456941+0000","last_change":"2024-02-07T12:22:07.450539+0000","last_active":"2024-02-07T12:22:33.456941+0000","last_peered":"2024-02-07T12:22:33.456941+0000","last_clean":"2024-02-07T12:22:33.456941+0000","last_became_active":"2024-02-07T12:22:07.450216+0000","last_became_peered":"2024-02-07T12:22:07.450216+0000","last_unstale":"2024-02-07T12:22:33.456941+0000","last_undegraded":"2024-02-07T12:22:33.456941+0000","last_fullsized":"2024-02-07T12:22:33.456941+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:14:33.069337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451879+0000","last_change":"2024-02-07T12:22:11.482109+0000","last_active":"2024-02-07T12:22:33.451879+0000","last_peered":"2024-02-07T12:22:33.451879+0000","last_clean":"2024-02-07T12:22:33.451879+0000","last_became_active":"2024-02-07T12:22:11.481555+0000","last_became_peered":"2024-02-07T12:22:11.481555+0000","last_unstale":"2024-02-07T12:22:33.451879+0000","last_undegraded":"2024-02-07T12:22:33.451879+0000","last_fullsized":"2024-02-07T12:22:33.451879+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:26:49.826771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479591+0000","last_change":"2024-02-07T12:22:09.464962+0000","last_active":"2024-02-07T12:22:40.479591+0000","last_peered":"2024-02-07T12:22:40.479591+0000","last_clean":"2024-02-07T12:22:40.479591+0000","last_became_active":"2024-02-07T12:22:09.464433+0000","last_became_peered":"2024-02-07T12:22:09.464433+0000","last_unstale":"2024-02-07T12:22:40.479591+0000","last_undegraded":"2024-02-07T12:22:40.479591+0000","last_fullsized":"2024-02-07T12:22:40.479591+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:08:52.172867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455940+0000","last_change":"2024-02-07T12:22:05.431234+0000","last_active":"2024-02-07T12:22:33.455940+0000","last_peered":"2024-02-07T12:22:33.455940+0000","last_clean":"2024-02-07T12:22:33.455940+0000","last_became_active":"2024-02-07T12:22:05.430815+0000","last_became_peered":"2024-02-07T12:22:05.430815+0000","last_unstale":"2024-02-07T12:22:33.455940+0000","last_undegraded":"2024-02-07T12:22:33.455940+0000","last_fullsized":"2024-02-07T12:22:33.455940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:34:58.630000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450373+0000","last_change":"2024-02-07T12:22:07.443330+0000","last_active":"2024-02-07T12:22:33.450373+0000","last_peered":"2024-02-07T12:22:33.450373+0000","last_clean":"2024-02-07T12:22:33.450373+0000","last_became_active":"2024-02-07T12:22:07.442911+0000","last_became_peered":"2024-02-07T12:22:07.442911+0000","last_unstale":"2024-02-07T12:22:33.450373+0000","last_undegraded":"2024-02-07T12:22:33.450373+0000","last_fullsized":"2024-02-07T12:22:33.450373+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:53:22.141512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451750+0000","last_change":"2024-02-07T12:22:11.481988+0000","last_active":"2024-02-07T12:22:33.451750+0000","last_peered":"2024-02-07T12:22:33.451750+0000","last_clean":"2024-02-07T12:22:33.451750+0000","last_became_active":"2024-02-07T12:22:11.481344+0000","last_became_peered":"2024-02-07T12:22:11.481344+0000","last_unstale":"2024-02-07T12:22:33.451750+0000","last_undegraded":"2024-02-07T12:22:33.451750+0000","last_fullsized":"2024-02-07T12:22:33.451750+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:34:34.290136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479667+0000","last_change":"2024-02-07T12:22:09.462773+0000","last_active":"2024-02-07T12:22:40.479667+0000","last_peered":"2024-02-07T12:22:40.479667+0000","last_clean":"2024-02-07T12:22:40.479667+0000","last_became_active":"2024-02-07T12:22:09.462241+0000","last_became_peered":"2024-02-07T12:22:09.462241+0000","last_unstale":"2024-02-07T12:22:40.479667+0000","last_undegraded":"2024-02-07T12:22:40.479667+0000","last_fullsized":"2024-02-07T12:22:40.479667+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:30:08.899685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455622+0000","last_change":"2024-02-07T12:22:05.435004+0000","last_active":"2024-02-07T12:22:33.455622+0000","last_peered":"2024-02-07T12:22:33.455622+0000","last_clean":"2024-02-07T12:22:33.455622+0000","last_became_active":"2024-02-07T12:22:05.432534+0000","last_became_peered":"2024-02-07T12:22:05.432534+0000","last_unstale":"2024-02-07T12:22:33.455622+0000","last_undegraded":"2024-02-07T12:22:33.455622+0000","last_fullsized":"2024-02-07T12:22:33.455622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:14:06.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451565+0000","last_change":"2024-02-07T12:22:07.446909+0000","last_active":"2024-02-07T12:22:33.451565+0000","last_peered":"2024-02-07T12:22:33.451565+0000","last_clean":"2024-02-07T12:22:33.451565+0000","last_became_active":"2024-02-07T12:22:07.445728+0000","last_became_peered":"2024-02-07T12:22:07.445728+0000","last_unstale":"2024-02-07T12:22:33.451565+0000","last_undegraded":"2024-02-07T12:22:33.451565+0000","last_fullsized":"2024-02-07T12:22:33.451565+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:31:36.456778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451257+0000","last_change":"2024-02-07T12:22:11.483311+0000","last_active":"2024-02-07T12:22:33.451257+0000","last_peered":"2024-02-07T12:22:33.451257+0000","last_clean":"2024-02-07T12:22:33.451257+0000","last_became_active":"2024-02-07T12:22:11.482210+0000","last_became_peered":"2024-02-07T12:22:11.482210+0000","last_unstale":"2024-02-07T12:22:33.451257+0000","last_undegraded":"2024-02-07T12:22:33.451257+0000","last_fullsized":"2024-02-07T12:22:33.451257+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:07:07.948765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453319+0000","last_change":"2024-02-07T12:22:09.465917+0000","last_active":"2024-02-07T12:22:33.453319+0000","last_peered":"2024-02-07T12:22:33.453319+0000","last_clean":"2024-02-07T12:22:33.453319+0000","last_became_active":"2024-02-07T12:22:09.465596+0000","last_became_peered":"2024-02-07T12:22:09.465596+0000","last_unstale":"2024-02-07T12:22:33.453319+0000","last_undegraded":"2024-02-07T12:22:33.453319+0000","last_fullsized":"2024-02-07T12:22:33.453319+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:35:08.970185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450619+0000","last_change":"2024-02-07T12:22:05.434175+0000","last_active":"2024-02-07T12:22:33.450619+0000","last_peered":"2024-02-07T12:22:33.450619+0000","last_clean":"2024-02-07T12:22:33.450619+0000","last_became_active":"2024-02-07T12:22:05.433784+0000","last_became_peered":"2024-02-07T12:22:05.433784+0000","last_unstale":"2024-02-07T12:22:33.450619+0000","last_undegraded":"2024-02-07T12:22:33.450619+0000","last_fullsized":"2024-02-07T12:22:33.450619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:21:04.038244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450543+0000","last_change":"2024-02-07T12:22:07.448016+0000","last_active":"2024-02-07T12:22:33.450543+0000","last_peered":"2024-02-07T12:22:33.450543+0000","last_clean":"2024-02-07T12:22:33.450543+0000","last_became_active":"2024-02-07T12:22:07.446717+0000","last_became_peered":"2024-02-07T12:22:07.446717+0000","last_unstale":"2024-02-07T12:22:33.450543+0000","last_undegraded":"2024-02-07T12:22:33.450543+0000","last_fullsized":"2024-02-07T12:22:33.450543+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:08:09.019276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453079+0000","last_change":"2024-02-07T12:22:11.485136+0000","last_active":"2024-02-07T12:22:33.453079+0000","last_peered":"2024-02-07T12:22:33.453079+0000","last_clean":"2024-02-07T12:22:33.453079+0000","last_became_active":"2024-02-07T12:22:11.484737+0000","last_became_peered":"2024-02-07T12:22:11.484737+0000","last_unstale":"2024-02-07T12:22:33.453079+0000","last_undegraded":"2024-02-07T12:22:33.453079+0000","last_fullsized":"2024-02-07T12:22:33.453079+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:14:39.741747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452373+0000","last_change":"2024-02-07T12:22:09.466941+0000","last_active":"2024-02-07T12:22:33.452373+0000","last_peered":"2024-02-07T12:22:33.452373+0000","last_clean":"2024-02-07T12:22:33.452373+0000","last_became_active":"2024-02-07T12:22:09.466585+0000","last_became_peered":"2024-02-07T12:22:09.466585+0000","last_unstale":"2024-02-07T12:22:33.452373+0000","last_undegraded":"2024-02-07T12:22:33.452373+0000","last_fullsized":"2024-02-07T12:22:33.452373+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:42:26.039043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454938+0000","last_change":"2024-02-07T12:22:05.435052+0000","last_active":"2024-02-07T12:22:33.454938+0000","last_peered":"2024-02-07T12:22:33.454938+0000","last_clean":"2024-02-07T12:22:33.454938+0000","last_became_active":"2024-02-07T12:22:05.434724+0000","last_became_peered":"2024-02-07T12:22:05.434724+0000","last_unstale":"2024-02-07T12:22:33.454938+0000","last_undegraded":"2024-02-07T12:22:33.454938+0000","last_fullsized":"2024-02-07T12:22:33.454938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:18:08.397410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454959+0000","last_change":"2024-02-07T12:22:07.449850+0000","last_active":"2024-02-07T12:22:33.454959+0000","last_peered":"2024-02-07T12:22:33.454959+0000","last_clean":"2024-02-07T12:22:33.454959+0000","last_became_active":"2024-02-07T12:22:07.449562+0000","last_became_peered":"2024-02-07T12:22:07.449562+0000","last_unstale":"2024-02-07T12:22:33.454959+0000","last_undegraded":"2024-02-07T12:22:33.454959+0000","last_fullsized":"2024-02-07T12:22:33.454959+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:56:16.044010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455265+0000","last_change":"2024-02-07T12:22:11.476414+0000","last_active":"2024-02-07T12:22:33.455265+0000","last_peered":"2024-02-07T12:22:33.455265+0000","last_clean":"2024-02-07T12:22:33.455265+0000","last_became_active":"2024-02-07T12:22:11.476127+0000","last_became_peered":"2024-02-07T12:22:11.476127+0000","last_unstale":"2024-02-07T12:22:33.455265+0000","last_undegraded":"2024-02-07T12:22:33.455265+0000","last_fullsized":"2024-02-07T12:22:33.455265+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:53:27.539410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456982+0000","last_change":"2024-02-07T12:22:05.434014+0000","last_active":"2024-02-07T12:22:33.456982+0000","last_peered":"2024-02-07T12:22:33.456982+0000","last_clean":"2024-02-07T12:22:33.456982+0000","last_became_active":"2024-02-07T12:22:05.432346+0000","last_became_peered":"2024-02-07T12:22:05.432346+0000","last_unstale":"2024-02-07T12:22:33.456982+0000","last_undegraded":"2024-02-07T12:22:33.456982+0000","last_fullsized":"2024-02-07T12:22:33.456982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:53:46.026960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456176+0000","last_change":"2024-02-07T12:22:07.450695+0000","last_active":"2024-02-07T12:22:33.456176+0000","last_peered":"2024-02-07T12:22:33.456176+0000","last_clean":"2024-02-07T12:22:33.456176+0000","last_became_active":"2024-02-07T12:22:07.450276+0000","last_became_peered":"2024-02-07T12:22:07.450276+0000","last_unstale":"2024-02-07T12:22:33.456176+0000","last_undegraded":"2024-02-07T12:22:33.456176+0000","last_fullsized":"2024-02-07T12:22:33.456176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:15:25.118040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479679+0000","last_change":"2024-02-07T12:22:09.463241+0000","last_active":"2024-02-07T12:22:40.479679+0000","last_peered":"2024-02-07T12:22:40.479679+0000","last_clean":"2024-02-07T12:22:40.479679+0000","last_became_active":"2024-02-07T12:22:09.462959+0000","last_became_peered":"2024-02-07T12:22:09.462959+0000","last_unstale":"2024-02-07T12:22:40.479679+0000","last_undegraded":"2024-02-07T12:22:40.479679+0000","last_fullsized":"2024-02-07T12:22:40.479679+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:45:32.777028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450939+0000","last_change":"2024-02-07T12:22:11.481803+0000","last_active":"2024-02-07T12:22:33.450939+0000","last_peered":"2024-02-07T12:22:33.450939+0000","last_clean":"2024-02-07T12:22:33.450939+0000","last_became_active":"2024-02-07T12:22:11.481491+0000","last_became_peered":"2024-02-07T12:22:11.481491+0000","last_unstale":"2024-02-07T12:22:33.450939+0000","last_undegraded":"2024-02-07T12:22:33.450939+0000","last_fullsized":"2024-02-07T12:22:33.450939+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:52:47.766315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452629+0000","last_change":"2024-02-07T12:22:05.437055+0000","last_active":"2024-02-07T12:22:33.452629+0000","last_peered":"2024-02-07T12:22:33.452629+0000","last_clean":"2024-02-07T12:22:33.452629+0000","last_became_active":"2024-02-07T12:22:05.436596+0000","last_became_peered":"2024-02-07T12:22:05.436596+0000","last_unstale":"2024-02-07T12:22:33.452629+0000","last_undegraded":"2024-02-07T12:22:33.452629+0000","last_fullsized":"2024-02-07T12:22:33.452629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:05:04.000703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479615+0000","last_change":"2024-02-07T12:22:07.441869+0000","last_active":"2024-02-07T12:22:40.479615+0000","last_peered":"2024-02-07T12:22:40.479615+0000","last_clean":"2024-02-07T12:22:40.479615+0000","last_became_active":"2024-02-07T12:22:07.441411+0000","last_became_peered":"2024-02-07T12:22:07.441411+0000","last_unstale":"2024-02-07T12:22:40.479615+0000","last_undegraded":"2024-02-07T12:22:40.479615+0000","last_fullsized":"2024-02-07T12:22:40.479615+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:01:00.861008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455519+0000","last_change":"2024-02-07T12:22:09.462832+0000","last_active":"2024-02-07T12:22:33.455519+0000","last_peered":"2024-02-07T12:22:33.455519+0000","last_clean":"2024-02-07T12:22:33.455519+0000","last_became_active":"2024-02-07T12:22:09.462387+0000","last_became_peered":"2024-02-07T12:22:09.462387+0000","last_unstale":"2024-02-07T12:22:33.455519+0000","last_undegraded":"2024-02-07T12:22:33.455519+0000","last_fullsized":"2024-02-07T12:22:33.455519+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:06:13.802167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450203+0000","last_change":"2024-02-07T12:22:11.484556+0000","last_active":"2024-02-07T12:22:33.450203+0000","last_peered":"2024-02-07T12:22:33.450203+0000","last_clean":"2024-02-07T12:22:33.450203+0000","last_became_active":"2024-02-07T12:22:11.483965+0000","last_became_peered":"2024-02-07T12:22:11.483965+0000","last_unstale":"2024-02-07T12:22:33.450203+0000","last_undegraded":"2024-02-07T12:22:33.450203+0000","last_fullsized":"2024-02-07T12:22:33.450203+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:29:18.294341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451517+0000","last_change":"2024-02-07T12:22:05.432148+0000","last_active":"2024-02-07T12:22:33.451517+0000","last_peered":"2024-02-07T12:22:33.451517+0000","last_clean":"2024-02-07T12:22:33.451517+0000","last_became_active":"2024-02-07T12:22:05.431868+0000","last_became_peered":"2024-02-07T12:22:05.431868+0000","last_unstale":"2024-02-07T12:22:33.451517+0000","last_undegraded":"2024-02-07T12:22:33.451517+0000","last_fullsized":"2024-02-07T12:22:33.451517+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:00:56.788222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452432+0000","last_change":"2024-02-07T12:22:07.447135+0000","last_active":"2024-02-07T12:22:33.452432+0000","last_peered":"2024-02-07T12:22:33.452432+0000","last_clean":"2024-02-07T12:22:33.452432+0000","last_became_active":"2024-02-07T12:22:07.445819+0000","last_became_peered":"2024-02-07T12:22:07.445819+0000","last_unstale":"2024-02-07T12:22:33.452432+0000","last_undegraded":"2024-02-07T12:22:33.452432+0000","last_fullsized":"2024-02-07T12:22:33.452432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:46:40.686303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451968+0000","last_change":"2024-02-07T12:22:09.465184+0000","last_active":"2024-02-07T12:22:33.451968+0000","last_peered":"2024-02-07T12:22:33.451968+0000","last_clean":"2024-02-07T12:22:33.451968+0000","last_became_active":"2024-02-07T12:22:09.464618+0000","last_became_peered":"2024-02-07T12:22:09.464618+0000","last_unstale":"2024-02-07T12:22:33.451968+0000","last_undegraded":"2024-02-07T12:22:33.451968+0000","last_fullsized":"2024-02-07T12:22:33.451968+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:23:44.574942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452664+0000","last_change":"2024-02-07T12:22:11.485373+0000","last_active":"2024-02-07T12:22:33.452664+0000","last_peered":"2024-02-07T12:22:33.452664+0000","last_clean":"2024-02-07T12:22:33.452664+0000","last_became_active":"2024-02-07T12:22:11.485039+0000","last_became_peered":"2024-02-07T12:22:11.485039+0000","last_unstale":"2024-02-07T12:22:33.452664+0000","last_undegraded":"2024-02-07T12:22:33.452664+0000","last_fullsized":"2024-02-07T12:22:33.452664+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:25:47.014293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454260+0000","last_change":"2024-02-07T12:22:07.450879+0000","last_active":"2024-02-07T12:22:33.454260+0000","last_peered":"2024-02-07T12:22:33.454260+0000","last_clean":"2024-02-07T12:22:33.454260+0000","last_became_active":"2024-02-07T12:22:07.450404+0000","last_became_peered":"2024-02-07T12:22:07.450404+0000","last_unstale":"2024-02-07T12:22:33.454260+0000","last_undegraded":"2024-02-07T12:22:33.454260+0000","last_fullsized":"2024-02-07T12:22:33.454260+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:42:37.924581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449887+0000","last_change":"2024-02-07T12:22:05.436755+0000","last_active":"2024-02-07T12:22:33.449887+0000","last_peered":"2024-02-07T12:22:33.449887+0000","last_clean":"2024-02-07T12:22:33.449887+0000","last_became_active":"2024-02-07T12:22:05.436477+0000","last_became_peered":"2024-02-07T12:22:05.436477+0000","last_unstale":"2024-02-07T12:22:33.449887+0000","last_undegraded":"2024-02-07T12:22:33.449887+0000","last_fullsized":"2024-02-07T12:22:33.449887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:45:36.025351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453532+0000","last_change":"2024-02-07T12:22:09.468042+0000","last_active":"2024-02-07T12:22:33.453532+0000","last_peered":"2024-02-07T12:22:33.453532+0000","last_clean":"2024-02-07T12:22:33.453532+0000","last_became_active":"2024-02-07T12:22:09.467600+0000","last_became_peered":"2024-02-07T12:22:09.467600+0000","last_unstale":"2024-02-07T12:22:33.453532+0000","last_undegraded":"2024-02-07T12:22:33.453532+0000","last_fullsized":"2024-02-07T12:22:33.453532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:33:43.797109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454611+0000","last_change":"2024-02-07T12:22:11.478779+0000","last_active":"2024-02-07T12:22:33.454611+0000","last_peered":"2024-02-07T12:22:33.454611+0000","last_clean":"2024-02-07T12:22:33.454611+0000","last_became_active":"2024-02-07T12:22:11.478510+0000","last_became_peered":"2024-02-07T12:22:11.478510+0000","last_unstale":"2024-02-07T12:22:33.454611+0000","last_undegraded":"2024-02-07T12:22:33.454611+0000","last_fullsized":"2024-02-07T12:22:33.454611+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:40:27.319317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450259+0000","last_change":"2024-02-07T12:22:07.443955+0000","last_active":"2024-02-07T12:22:33.450259+0000","last_peered":"2024-02-07T12:22:33.450259+0000","last_clean":"2024-02-07T12:22:33.450259+0000","last_became_active":"2024-02-07T12:22:07.443467+0000","last_became_peered":"2024-02-07T12:22:07.443467+0000","last_unstale":"2024-02-07T12:22:33.450259+0000","last_undegraded":"2024-02-07T12:22:33.450259+0000","last_fullsized":"2024-02-07T12:22:33.450259+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:04:42.487324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451693+0000","last_change":"2024-02-07T12:22:05.431585+0000","last_active":"2024-02-07T12:22:33.451693+0000","last_peered":"2024-02-07T12:22:33.451693+0000","last_clean":"2024-02-07T12:22:33.451693+0000","last_became_active":"2024-02-07T12:22:05.431178+0000","last_became_peered":"2024-02-07T12:22:05.431178+0000","last_unstale":"2024-02-07T12:22:33.451693+0000","last_undegraded":"2024-02-07T12:22:33.451693+0000","last_fullsized":"2024-02-07T12:22:33.451693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:22:50.924026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:35.566163+0000","last_change":"2024-02-07T12:21:51.878602+0000","last_active":"2024-02-07T12:22:35.566163+0000","last_peered":"2024-02-07T12:22:35.566163+0000","last_clean":"2024-02-07T12:22:35.566163+0000","last_became_active":"2024-02-07T12:21:51.877629+0000","last_became_peered":"2024-02-07T12:21:51.877629+0000","last_unstale":"2024-02-07T12:22:35.566163+0000","last_undegraded":"2024-02-07T12:22:35.566163+0000","last_fullsized":"2024-02-07T12:22:35.566163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:21:50.856971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:21:50.856971+0000","last_clean_scrub_stamp":"2024-02-07T12:21:50.856971+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:56:58.825088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453141+0000","last_change":"2024-02-07T12:22:09.461339+0000","last_active":"2024-02-07T12:22:33.453141+0000","last_peered":"2024-02-07T12:22:33.453141+0000","last_clean":"2024-02-07T12:22:33.453141+0000","last_became_active":"2024-02-07T12:22:09.461004+0000","last_became_peered":"2024-02-07T12:22:09.461004+0000","last_unstale":"2024-02-07T12:22:33.453141+0000","last_undegraded":"2024-02-07T12:22:33.453141+0000","last_fullsized":"2024-02-07T12:22:33.453141+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:48:49.705656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451668+0000","last_change":"2024-02-07T12:22:11.481196+0000","last_active":"2024-02-07T12:22:33.451668+0000","last_peered":"2024-02-07T12:22:33.451668+0000","last_clean":"2024-02-07T12:22:33.451668+0000","last_became_active":"2024-02-07T12:22:11.480340+0000","last_became_peered":"2024-02-07T12:22:11.480340+0000","last_unstale":"2024-02-07T12:22:33.451668+0000","last_undegraded":"2024-02-07T12:22:33.451668+0000","last_fullsized":"2024-02-07T12:22:33.451668+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:34:57.392750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449968+0000","last_change":"2024-02-07T12:22:07.447297+0000","last_active":"2024-02-07T12:22:33.449968+0000","last_peered":"2024-02-07T12:22:33.449968+0000","last_clean":"2024-02-07T12:22:33.449968+0000","last_became_active":"2024-02-07T12:22:07.446966+0000","last_became_peered":"2024-02-07T12:22:07.446966+0000","last_unstale":"2024-02-07T12:22:33.449968+0000","last_undegraded":"2024-02-07T12:22:33.449968+0000","last_fullsized":"2024-02-07T12:22:33.449968+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:34:39.500538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452584+0000","last_change":"2024-02-07T12:22:05.436374+0000","last_active":"2024-02-07T12:22:33.452584+0000","last_peered":"2024-02-07T12:22:33.452584+0000","last_clean":"2024-02-07T12:22:33.452584+0000","last_became_active":"2024-02-07T12:22:05.436047+0000","last_became_peered":"2024-02-07T12:22:05.436047+0000","last_unstale":"2024-02-07T12:22:33.452584+0000","last_undegraded":"2024-02-07T12:22:33.452584+0000","last_fullsized":"2024-02-07T12:22:33.452584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:00:44.000402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455831+0000","last_change":"2024-02-07T12:22:09.465521+0000","last_active":"2024-02-07T12:22:33.455831+0000","last_peered":"2024-02-07T12:22:33.455831+0000","last_clean":"2024-02-07T12:22:33.455831+0000","last_became_active":"2024-02-07T12:22:09.464639+0000","last_became_peered":"2024-02-07T12:22:09.464639+0000","last_unstale":"2024-02-07T12:22:33.455831+0000","last_undegraded":"2024-02-07T12:22:33.455831+0000","last_fullsized":"2024-02-07T12:22:33.455831+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:21:01.449598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451068+0000","last_change":"2024-02-07T12:22:11.481507+0000","last_active":"2024-02-07T12:22:33.451068+0000","last_peered":"2024-02-07T12:22:33.451068+0000","last_clean":"2024-02-07T12:22:33.451068+0000","last_became_active":"2024-02-07T12:22:11.480608+0000","last_became_peered":"2024-02-07T12:22:11.480608+0000","last_unstale":"2024-02-07T12:22:33.451068+0000","last_undegraded":"2024-02-07T12:22:33.451068+0000","last_fullsized":"2024-02-07T12:22:33.451068+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:58:46.429737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.457057+0000","last_change":"2024-02-07T12:22:05.432607+0000","last_active":"2024-02-07T12:22:33.457057+0000","last_peered":"2024-02-07T12:22:33.457057+0000","last_clean":"2024-02-07T12:22:33.457057+0000","last_became_active":"2024-02-07T12:22:05.431011+0000","last_became_peered":"2024-02-07T12:22:05.431011+0000","last_unstale":"2024-02-07T12:22:33.457057+0000","last_undegraded":"2024-02-07T12:22:33.457057+0000","last_fullsized":"2024-02-07T12:22:33.457057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:54:54.832166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":80,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479384+0000","last_change":"2024-02-07T12:22:07.444110+0000","last_active":"2024-02-07T12:22:40.479384+0000","last_peered":"2024-02-07T12:22:40.479384+0000","last_clean":"2024-02-07T12:22:40.479384+0000","last_became_active":"2024-02-07T12:22:07.443639+0000","last_became_peered":"2024-02-07T12:22:07.443639+0000","last_unstale":"2024-02-07T12:22:40.479384+0000","last_undegraded":"2024-02-07T12:22:40.479384+0000","last_fullsized":"2024-02-07T12:22:40.479384+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:13:24.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479606+0000","last_change":"2024-02-07T12:22:09.465667+0000","last_active":"2024-02-07T12:22:40.479606+0000","last_peered":"2024-02-07T12:22:40.479606+0000","last_clean":"2024-02-07T12:22:40.479606+0000","last_became_active":"2024-02-07T12:22:09.464933+0000","last_became_peered":"2024-02-07T12:22:09.464933+0000","last_unstale":"2024-02-07T12:22:40.479606+0000","last_undegraded":"2024-02-07T12:22:40.479606+0000","last_fullsized":"2024-02-07T12:22:40.479606+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:46:51.835370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450285+0000","last_change":"2024-02-07T12:22:11.482271+0000","last_active":"2024-02-07T12:22:33.450285+0000","last_peered":"2024-02-07T12:22:33.450285+0000","last_clean":"2024-02-07T12:22:33.450285+0000","last_became_active":"2024-02-07T12:22:11.481514+0000","last_became_peered":"2024-02-07T12:22:11.481514+0000","last_unstale":"2024-02-07T12:22:33.450285+0000","last_undegraded":"2024-02-07T12:22:33.450285+0000","last_fullsized":"2024-02-07T12:22:33.450285+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:32:36.638540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452574+0000","last_change":"2024-02-07T12:22:05.436878+0000","last_active":"2024-02-07T12:22:33.452574+0000","last_peered":"2024-02-07T12:22:33.452574+0000","last_clean":"2024-02-07T12:22:33.452574+0000","last_became_active":"2024-02-07T12:22:05.436605+0000","last_became_peered":"2024-02-07T12:22:05.436605+0000","last_unstale":"2024-02-07T12:22:33.452574+0000","last_undegraded":"2024-02-07T12:22:33.452574+0000","last_fullsized":"2024-02-07T12:22:33.452574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:34:00.686024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453753+0000","last_change":"2024-02-07T12:22:07.451080+0000","last_active":"2024-02-07T12:22:33.453753+0000","last_peered":"2024-02-07T12:22:33.453753+0000","last_clean":"2024-02-07T12:22:33.453753+0000","last_became_active":"2024-02-07T12:22:07.450700+0000","last_became_peered":"2024-02-07T12:22:07.450700+0000","last_unstale":"2024-02-07T12:22:33.453753+0000","last_undegraded":"2024-02-07T12:22:33.453753+0000","last_fullsized":"2024-02-07T12:22:33.453753+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:11:00.332249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479383+0000","last_change":"2024-02-07T12:22:09.467882+0000","last_active":"2024-02-07T12:22:40.479383+0000","last_peered":"2024-02-07T12:22:40.479383+0000","last_clean":"2024-02-07T12:22:40.479383+0000","last_became_active":"2024-02-07T12:22:09.467307+0000","last_became_peered":"2024-02-07T12:22:09.467307+0000","last_unstale":"2024-02-07T12:22:40.479383+0000","last_undegraded":"2024-02-07T12:22:40.479383+0000","last_fullsized":"2024-02-07T12:22:40.479383+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:41:49.322998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455223+0000","last_change":"2024-02-07T12:22:11.476130+0000","last_active":"2024-02-07T12:22:33.455223+0000","last_peered":"2024-02-07T12:22:33.455223+0000","last_clean":"2024-02-07T12:22:33.455223+0000","last_became_active":"2024-02-07T12:22:11.475838+0000","last_became_peered":"2024-02-07T12:22:11.475838+0000","last_unstale":"2024-02-07T12:22:33.455223+0000","last_undegraded":"2024-02-07T12:22:33.455223+0000","last_fullsized":"2024-02-07T12:22:33.455223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:48:12.085334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453004+0000","last_change":"2024-02-07T12:22:05.435540+0000","last_active":"2024-02-07T12:22:33.453004+0000","last_peered":"2024-02-07T12:22:33.453004+0000","last_clean":"2024-02-07T12:22:33.453004+0000","last_became_active":"2024-02-07T12:22:05.435182+0000","last_became_peered":"2024-02-07T12:22:05.435182+0000","last_unstale":"2024-02-07T12:22:33.453004+0000","last_undegraded":"2024-02-07T12:22:33.453004+0000","last_fullsized":"2024-02-07T12:22:33.453004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:09:58.516333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452888+0000","last_change":"2024-02-07T12:22:07.451226+0000","last_active":"2024-02-07T12:22:33.452888+0000","last_peered":"2024-02-07T12:22:33.452888+0000","last_clean":"2024-02-07T12:22:33.452888+0000","last_became_active":"2024-02-07T12:22:07.450828+0000","last_became_peered":"2024-02-07T12:22:07.450828+0000","last_unstale":"2024-02-07T12:22:33.452888+0000","last_undegraded":"2024-02-07T12:22:33.452888+0000","last_fullsized":"2024-02-07T12:22:33.452888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:45:47.115738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454658+0000","last_change":"2024-02-07T12:22:09.464609+0000","last_active":"2024-02-07T12:22:33.454658+0000","last_peered":"2024-02-07T12:22:33.454658+0000","last_clean":"2024-02-07T12:22:33.454658+0000","last_became_active":"2024-02-07T12:22:09.463612+0000","last_became_peered":"2024-02-07T12:22:09.463612+0000","last_unstale":"2024-02-07T12:22:33.454658+0000","last_undegraded":"2024-02-07T12:22:33.454658+0000","last_fullsized":"2024-02-07T12:22:33.454658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:49:10.287793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452813+0000","last_change":"2024-02-07T12:22:11.480663+0000","last_active":"2024-02-07T12:22:33.452813+0000","last_peered":"2024-02-07T12:22:33.452813+0000","last_clean":"2024-02-07T12:22:33.452813+0000","last_became_active":"2024-02-07T12:22:11.480316+0000","last_became_peered":"2024-02-07T12:22:11.480316+0000","last_unstale":"2024-02-07T12:22:33.452813+0000","last_undegraded":"2024-02-07T12:22:33.452813+0000","last_fullsized":"2024-02-07T12:22:33.452813+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:12:22.960223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479623+0000","last_change":"2024-02-07T12:22:09.463477+0000","last_active":"2024-02-07T12:22:40.479623+0000","last_peered":"2024-02-07T12:22:40.479623+0000","last_clean":"2024-02-07T12:22:40.479623+0000","last_became_active":"2024-02-07T12:22:09.463165+0000","last_became_peered":"2024-02-07T12:22:09.463165+0000","last_unstale":"2024-02-07T12:22:40.479623+0000","last_undegraded":"2024-02-07T12:22:40.479623+0000","last_fullsized":"2024-02-07T12:22:40.479623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:30:24.481576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450337+0000","last_change":"2024-02-07T12:22:05.436478+0000","last_active":"2024-02-07T12:22:33.450337+0000","last_peered":"2024-02-07T12:22:33.450337+0000","last_clean":"2024-02-07T12:22:33.450337+0000","last_became_active":"2024-02-07T12:22:05.436198+0000","last_became_peered":"2024-02-07T12:22:05.436198+0000","last_unstale":"2024-02-07T12:22:33.450337+0000","last_undegraded":"2024-02-07T12:22:33.450337+0000","last_fullsized":"2024-02-07T12:22:33.450337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:36:44.041796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455141+0000","last_change":"2024-02-07T12:22:07.449899+0000","last_active":"2024-02-07T12:22:33.455141+0000","last_peered":"2024-02-07T12:22:33.455141+0000","last_clean":"2024-02-07T12:22:33.455141+0000","last_became_active":"2024-02-07T12:22:07.449675+0000","last_became_peered":"2024-02-07T12:22:07.449675+0000","last_unstale":"2024-02-07T12:22:33.455141+0000","last_undegraded":"2024-02-07T12:22:33.455141+0000","last_fullsized":"2024-02-07T12:22:33.455141+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:15:17.745640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456918+0000","last_change":"2024-02-07T12:22:11.478981+0000","last_active":"2024-02-07T12:22:33.456918+0000","last_peered":"2024-02-07T12:22:33.456918+0000","last_clean":"2024-02-07T12:22:33.456918+0000","last_became_active":"2024-02-07T12:22:11.478681+0000","last_became_peered":"2024-02-07T12:22:11.478681+0000","last_unstale":"2024-02-07T12:22:33.456918+0000","last_undegraded":"2024-02-07T12:22:33.456918+0000","last_fullsized":"2024-02-07T12:22:33.456918+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:18:24.259519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453390+0000","last_change":"2024-02-07T12:22:09.466184+0000","last_active":"2024-02-07T12:22:33.453390+0000","last_peered":"2024-02-07T12:22:33.453390+0000","last_clean":"2024-02-07T12:22:33.453390+0000","last_became_active":"2024-02-07T12:22:09.465747+0000","last_became_peered":"2024-02-07T12:22:09.465747+0000","last_unstale":"2024-02-07T12:22:33.453390+0000","last_undegraded":"2024-02-07T12:22:33.453390+0000","last_fullsized":"2024-02-07T12:22:33.453390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:55:00.286671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450242+0000","last_change":"2024-02-07T12:22:05.437136+0000","last_active":"2024-02-07T12:22:33.450242+0000","last_peered":"2024-02-07T12:22:33.450242+0000","last_clean":"2024-02-07T12:22:33.450242+0000","last_became_active":"2024-02-07T12:22:05.436856+0000","last_became_peered":"2024-02-07T12:22:05.436856+0000","last_unstale":"2024-02-07T12:22:33.450242+0000","last_undegraded":"2024-02-07T12:22:33.450242+0000","last_fullsized":"2024-02-07T12:22:33.450242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:37:36.067961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.454962+0000","last_change":"2024-02-07T12:22:07.450981+0000","last_active":"2024-02-07T12:22:33.454962+0000","last_peered":"2024-02-07T12:22:33.454962+0000","last_clean":"2024-02-07T12:22:33.454962+0000","last_became_active":"2024-02-07T12:22:07.450728+0000","last_became_peered":"2024-02-07T12:22:07.450728+0000","last_unstale":"2024-02-07T12:22:33.454962+0000","last_undegraded":"2024-02-07T12:22:33.454962+0000","last_fullsized":"2024-02-07T12:22:33.454962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:11:21.727261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455749+0000","last_change":"2024-02-07T12:22:11.478740+0000","last_active":"2024-02-07T12:22:33.455749+0000","last_peered":"2024-02-07T12:22:33.455749+0000","last_clean":"2024-02-07T12:22:33.455749+0000","last_became_active":"2024-02-07T12:22:11.478446+0000","last_became_peered":"2024-02-07T12:22:11.478446+0000","last_unstale":"2024-02-07T12:22:33.455749+0000","last_undegraded":"2024-02-07T12:22:33.455749+0000","last_fullsized":"2024-02-07T12:22:33.455749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:52:17.097751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456010+0000","last_change":"2024-02-07T12:22:05.432895+0000","last_active":"2024-02-07T12:22:33.456010+0000","last_peered":"2024-02-07T12:22:33.456010+0000","last_clean":"2024-02-07T12:22:33.456010+0000","last_became_active":"2024-02-07T12:22:05.431968+0000","last_became_peered":"2024-02-07T12:22:05.431968+0000","last_unstale":"2024-02-07T12:22:33.456010+0000","last_undegraded":"2024-02-07T12:22:33.456010+0000","last_fullsized":"2024-02-07T12:22:33.456010+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:49:33.225778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451597+0000","last_change":"2024-02-07T12:22:07.443455+0000","last_active":"2024-02-07T12:22:33.451597+0000","last_peered":"2024-02-07T12:22:33.451597+0000","last_clean":"2024-02-07T12:22:33.451597+0000","last_became_active":"2024-02-07T12:22:07.443079+0000","last_became_peered":"2024-02-07T12:22:07.443079+0000","last_unstale":"2024-02-07T12:22:33.451597+0000","last_undegraded":"2024-02-07T12:22:33.451597+0000","last_fullsized":"2024-02-07T12:22:33.451597+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:12:14.809707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451233+0000","last_change":"2024-02-07T12:22:09.461182+0000","last_active":"2024-02-07T12:22:33.451233+0000","last_peered":"2024-02-07T12:22:33.451233+0000","last_clean":"2024-02-07T12:22:33.451233+0000","last_became_active":"2024-02-07T12:22:09.460804+0000","last_became_peered":"2024-02-07T12:22:09.460804+0000","last_unstale":"2024-02-07T12:22:33.451233+0000","last_undegraded":"2024-02-07T12:22:33.451233+0000","last_fullsized":"2024-02-07T12:22:33.451233+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:44:49.012329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451588+0000","last_change":"2024-02-07T12:22:11.481792+0000","last_active":"2024-02-07T12:22:33.451588+0000","last_peered":"2024-02-07T12:22:33.451588+0000","last_clean":"2024-02-07T12:22:33.451588+0000","last_became_active":"2024-02-07T12:22:11.481457+0000","last_became_peered":"2024-02-07T12:22:11.481457+0000","last_unstale":"2024-02-07T12:22:33.451588+0000","last_undegraded":"2024-02-07T12:22:33.451588+0000","last_fullsized":"2024-02-07T12:22:33.451588+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:07:36.450089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:40.479615+0000","last_change":"2024-02-07T12:22:09.463439+0000","last_active":"2024-02-07T12:22:40.479615+0000","last_peered":"2024-02-07T12:22:40.479615+0000","last_clean":"2024-02-07T12:22:40.479615+0000","last_became_active":"2024-02-07T12:22:09.463107+0000","last_became_peered":"2024-02-07T12:22:09.463107+0000","last_unstale":"2024-02-07T12:22:40.479615+0000","last_undegraded":"2024-02-07T12:22:40.479615+0000","last_fullsized":"2024-02-07T12:22:40.479615+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:36:40.943193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456761+0000","last_change":"2024-02-07T12:22:05.434900+0000","last_active":"2024-02-07T12:22:33.456761+0000","last_peered":"2024-02-07T12:22:33.456761+0000","last_clean":"2024-02-07T12:22:33.456761+0000","last_became_active":"2024-02-07T12:22:05.432480+0000","last_became_peered":"2024-02-07T12:22:05.432480+0000","last_unstale":"2024-02-07T12:22:33.456761+0000","last_undegraded":"2024-02-07T12:22:33.456761+0000","last_fullsized":"2024-02-07T12:22:33.456761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:52:58.113664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450908+0000","last_change":"2024-02-07T12:22:07.445952+0000","last_active":"2024-02-07T12:22:33.450908+0000","last_peered":"2024-02-07T12:22:33.450908+0000","last_clean":"2024-02-07T12:22:33.450908+0000","last_became_active":"2024-02-07T12:22:07.445656+0000","last_became_peered":"2024-02-07T12:22:07.445656+0000","last_unstale":"2024-02-07T12:22:33.450908+0000","last_undegraded":"2024-02-07T12:22:33.450908+0000","last_fullsized":"2024-02-07T12:22:33.450908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:09:09.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455120+0000","last_change":"2024-02-07T12:22:11.476031+0000","last_active":"2024-02-07T12:22:33.455120+0000","last_peered":"2024-02-07T12:22:33.455120+0000","last_clean":"2024-02-07T12:22:33.455120+0000","last_became_active":"2024-02-07T12:22:11.475699+0000","last_became_peered":"2024-02-07T12:22:11.475699+0000","last_unstale":"2024-02-07T12:22:33.455120+0000","last_undegraded":"2024-02-07T12:22:33.455120+0000","last_fullsized":"2024-02-07T12:22:33.455120+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:26:07.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455475+0000","last_change":"2024-02-07T12:22:05.430181+0000","last_active":"2024-02-07T12:22:33.455475+0000","last_peered":"2024-02-07T12:22:33.455475+0000","last_clean":"2024-02-07T12:22:33.455475+0000","last_became_active":"2024-02-07T12:22:05.429629+0000","last_became_peered":"2024-02-07T12:22:05.429629+0000","last_unstale":"2024-02-07T12:22:33.455475+0000","last_undegraded":"2024-02-07T12:22:33.455475+0000","last_fullsized":"2024-02-07T12:22:33.455475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:38:11.319198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453308+0000","last_change":"2024-02-07T12:22:07.442136+0000","last_active":"2024-02-07T12:22:33.453308+0000","last_peered":"2024-02-07T12:22:33.453308+0000","last_clean":"2024-02-07T12:22:33.453308+0000","last_became_active":"2024-02-07T12:22:07.441582+0000","last_became_peered":"2024-02-07T12:22:07.441582+0000","last_unstale":"2024-02-07T12:22:33.453308+0000","last_undegraded":"2024-02-07T12:22:33.453308+0000","last_fullsized":"2024-02-07T12:22:33.453308+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:50:34.243579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450722+0000","last_change":"2024-02-07T12:22:09.463092+0000","last_active":"2024-02-07T12:22:33.450722+0000","last_peered":"2024-02-07T12:22:33.450722+0000","last_clean":"2024-02-07T12:22:33.450722+0000","last_became_active":"2024-02-07T12:22:09.462724+0000","last_became_peered":"2024-02-07T12:22:09.462724+0000","last_unstale":"2024-02-07T12:22:33.450722+0000","last_undegraded":"2024-02-07T12:22:33.450722+0000","last_fullsized":"2024-02-07T12:22:33.450722+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:08:56.226751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.449758+0000","last_change":"2024-02-07T12:22:11.482625+0000","last_active":"2024-02-07T12:22:33.449758+0000","last_peered":"2024-02-07T12:22:33.449758+0000","last_clean":"2024-02-07T12:22:33.449758+0000","last_became_active":"2024-02-07T12:22:11.482329+0000","last_became_peered":"2024-02-07T12:22:11.482329+0000","last_unstale":"2024-02-07T12:22:33.449758+0000","last_undegraded":"2024-02-07T12:22:33.449758+0000","last_fullsized":"2024-02-07T12:22:33.449758+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:20:04.028005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.456118+0000","last_change":"2024-02-07T12:22:05.430903+0000","last_active":"2024-02-07T12:22:33.456118+0000","last_peered":"2024-02-07T12:22:33.456118+0000","last_clean":"2024-02-07T12:22:33.456118+0000","last_became_active":"2024-02-07T12:22:05.430618+0000","last_became_peered":"2024-02-07T12:22:05.430618+0000","last_unstale":"2024-02-07T12:22:33.456118+0000","last_undegraded":"2024-02-07T12:22:33.456118+0000","last_fullsized":"2024-02-07T12:22:33.456118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T12:53:58.556906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451413+0000","last_change":"2024-02-07T12:22:07.447445+0000","last_active":"2024-02-07T12:22:33.451413+0000","last_peered":"2024-02-07T12:22:33.451413+0000","last_clean":"2024-02-07T12:22:33.451413+0000","last_became_active":"2024-02-07T12:22:07.445865+0000","last_became_peered":"2024-02-07T12:22:07.445865+0000","last_unstale":"2024-02-07T12:22:33.451413+0000","last_undegraded":"2024-02-07T12:22:33.451413+0000","last_fullsized":"2024-02-07T12:22:33.451413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:57:00.770216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.451186+0000","last_change":"2024-02-07T12:22:09.465013+0000","last_active":"2024-02-07T12:22:33.451186+0000","last_peered":"2024-02-07T12:22:33.451186+0000","last_clean":"2024-02-07T12:22:33.451186+0000","last_became_active":"2024-02-07T12:22:09.464676+0000","last_became_peered":"2024-02-07T12:22:09.464676+0000","last_unstale":"2024-02-07T12:22:33.451186+0000","last_undegraded":"2024-02-07T12:22:33.451186+0000","last_fullsized":"2024-02-07T12:22:33.451186+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:35:34.706682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.450470+0000","last_change":"2024-02-07T12:22:11.482238+0000","last_active":"2024-02-07T12:22:33.450470+0000","last_peered":"2024-02-07T12:22:33.450470+0000","last_clean":"2024-02-07T12:22:33.450470+0000","last_became_active":"2024-02-07T12:22:11.480961+0000","last_became_peered":"2024-02-07T12:22:11.480961+0000","last_unstale":"2024-02-07T12:22:33.450470+0000","last_undegraded":"2024-02-07T12:22:33.450470+0000","last_fullsized":"2024-02-07T12:22:33.450470+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:10.457377+0000","last_clean_scrub_stamp":"2024-02-07T12:22:10.457377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:00:17.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.452795+0000","last_change":"2024-02-07T12:22:05.433668+0000","last_active":"2024-02-07T12:22:33.452795+0000","last_peered":"2024-02-07T12:22:33.452795+0000","last_clean":"2024-02-07T12:22:33.452795+0000","last_became_active":"2024-02-07T12:22:05.433406+0000","last_became_peered":"2024-02-07T12:22:05.433406+0000","last_unstale":"2024-02-07T12:22:33.452795+0000","last_undegraded":"2024-02-07T12:22:33.452795+0000","last_fullsized":"2024-02-07T12:22:33.452795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:04.415789+0000","last_clean_scrub_stamp":"2024-02-07T12:22:04.415789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:22:41.076847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.453097+0000","last_change":"2024-02-07T12:22:07.442138+0000","last_active":"2024-02-07T12:22:33.453097+0000","last_peered":"2024-02-07T12:22:33.453097+0000","last_clean":"2024-02-07T12:22:33.453097+0000","last_became_active":"2024-02-07T12:22:07.441874+0000","last_became_peered":"2024-02-07T12:22:07.441874+0000","last_unstale":"2024-02-07T12:22:33.453097+0000","last_undegraded":"2024-02-07T12:22:33.453097+0000","last_fullsized":"2024-02-07T12:22:33.453097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:06.429783+0000","last_clean_scrub_stamp":"2024-02-07T12:22:06.429783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:29:13.828193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T12:22:33.455128+0000","last_change":"2024-02-07T12:22:09.465552+0000","last_active":"2024-02-07T12:22:33.455128+0000","last_peered":"2024-02-07T12:22:33.455128+0000","last_clean":"2024-02-07T12:22:33.455128+0000","last_became_active":"2024-02-07T12:22:09.464794+0000","last_became_peered":"2024-02-07T12:22:09.464794+0000","last_unstale":"2024-02-07T12:22:33.455128+0000","last_undegraded":"2024-02-07T12:22:33.455128+0000","last_fullsized":"2024-02-07T12:22:33.455128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T12:22:08.442029+0000","last_clean_scrub_stamp":"2024-02-07T12:22:08.442029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:04:40.717522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411343,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738392,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 7 12:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 7 12:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.41799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-07T12:22:53.550 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-07T12:22:53.550 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-07T12:22:53.550 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-07T12:22:53.551 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- ceph health --format=json 2024-02-07T12:22:53.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:53 smithi171 ceph-mon[30005]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:53.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:53 smithi171 ceph-mon[30005]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:22:53.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:53 smithi106 ceph-mon[29693]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:53.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:53 smithi106 ceph-mon[29693]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T12:22:55.552 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:55.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:55 smithi171 ceph-mon[30005]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:55.670 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:55 smithi078 ceph-mon[25576]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:55.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:55 smithi106 ceph-mon[29693]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:57.261 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 podman[53651]: 2024-02-07 12:22:57.075539776 +0000 UTC m=+0.218007295 container create 135d6b122780e3f66c4058a73f0927b036523263884a667eab03e67c7f57a899 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T12:22:57.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-mon[25576]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:57.619 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 podman[53651]: 2024-02-07 12:22:57.361508117 +0000 UTC m=+0.503975635 container init 135d6b122780e3f66c4058a73f0927b036523263884a667eab03e67c7f57a899 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T12:22:57.619 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 podman[53651]: 2024-02-07 12:22:57.367330308 +0000 UTC m=+0.509797828 container start 135d6b122780e3f66c4058a73f0927b036523263884a667eab03e67c7f57a899 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T12:22:57.619 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:22:57.380Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-07T12:22:57.619 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:22:57.380Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-07T12:22:57.619 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:22:57.381Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.78 port=9094 2024-02-07T12:22:57.620 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:22:57.382Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-07T12:22:57.620 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:22:57.405Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-07T12:22:57.620 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:22:57.406Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-07T12:22:57.620 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:22:57.408Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-07T12:22:57.620 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:22:57.408Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-07T12:22:57.620 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 bash[53651]: 135d6b122780e3f66c4058a73f0927b036523263884a667eab03e67c7f57a899 2024-02-07T12:22:57.620 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:57 smithi078 systemd[1]: Started Ceph alertmanager.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:22:57.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:57 smithi171 ceph-mon[30005]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:57.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:57 smithi106 ceph-mon[29693]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:57.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:22:57] ENGINE Bus STOPPING 2024-02-07T12:22:57.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:22:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-07T12:22:57.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:22:57] ENGINE Bus STOPPED 2024-02-07T12:22:57.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:22:57] ENGINE Bus STARTING 2024-02-07T12:22:57.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:22:57] ENGINE Serving on http://:::9283 2024-02-07T12:22:57.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:22:57 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:22:57] ENGINE Bus STARTED 2024-02-07T12:22:58.283 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-07T12:22:58.283 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-07T12:22:58.604 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:58.604 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.604 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.604 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.604 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.605 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-07T12:22:58.605 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.605 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.605 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T12:22:58.605 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T12:22:58.605 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.605 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: Deploying daemon grafana.a on smithi106 2024-02-07T12:22:58.605 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:58 smithi078 ceph-mon[25576]: from='client.? 172.21.15.78:0/3753577994' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-07T12:22:58.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:58.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: Deploying daemon grafana.a on smithi106 2024-02-07T12:22:58.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:58 smithi106 ceph-mon[29693]: from='client.? 172.21.15.78:0/3753577994' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-07T12:22:58.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:22:58.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-07T12:22:58.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T12:22:58.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T12:22:58.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:58.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: Deploying daemon grafana.a on smithi106 2024-02-07T12:22:58.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:58 smithi171 ceph-mon[30005]: from='client.? 172.21.15.78:0/3753577994' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-07T12:22:58.922 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-07T12:22:58.922 INFO:tasks.cephadm:Setup complete, yielding 2024-02-07T12:22:58.923 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-07T12:22:58.933 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-02-07T12:22:58.934 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-02-07T12:22:59.170 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:22:59.557 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:22:59 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:22:59.557 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:22:59 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:22:59.383Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000902782s 2024-02-07T12:22:59.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:22:59 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:00.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:22:59 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:00.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:00 smithi078 ceph-mon[25576]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout:167 167 2024-02-07T12:23:00.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:00 smithi171 ceph-mon[30005]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:01.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:00 smithi106 ceph-mon[29693]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:01.438 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- bash -c 'ceph orch status' 2024-02-07T12:23:01.674 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:23:02.381 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:02 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ::ffff:172.21.15.106 - - [07/Feb/2024:12:23:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T12:23:03.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:03 smithi078 ceph-mon[25576]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:03.662 INFO:teuthology.orchestra.run.smithi078.stdout:Backend: cephadm 2024-02-07T12:23:03.662 INFO:teuthology.orchestra.run.smithi078.stdout:Available: Yes 2024-02-07T12:23:03.662 INFO:teuthology.orchestra.run.smithi078.stdout:Paused: No 2024-02-07T12:23:03.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:03 smithi171 ceph-mon[30005]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:03.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:03 smithi106 ceph-mon[29693]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:04.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-07T12:23:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:23:04.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:04 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:23:04.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:04 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:23:04.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:04 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:23:05.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:05 smithi171 ceph-mon[30005]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:05.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:05 smithi171 ceph-mon[30005]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:05.689 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:05 smithi078 ceph-mon[25576]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:05.689 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:05 smithi078 ceph-mon[25576]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:05.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:05 smithi106 ceph-mon[29693]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:05.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:05 smithi106 ceph-mon[29693]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:06.396 INFO:teuthology.orchestra.run.smithi078.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-07T12:23:06.396 INFO:teuthology.orchestra.run.smithi078.stdout:alertmanager.a smithi078 *:9093,9094 starting - - - - 2024-02-07T12:23:06.396 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.x smithi106 *:8443,8765 running (2m) 29s ago 2m 447M - 19.0.0-1252-g44458cce 79957b372307 3deecab659e4 2024-02-07T12:23:06.396 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.y smithi078 *:9283,8765 running (4m) 23s ago 4m 482M - 19.0.0-1252-g44458cce 79957b372307 fa19eb12e0b5 2024-02-07T12:23:06.397 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.z smithi171 *:8443,8765 running (2m) 31s ago 2m 447M - 19.0.0-1252-g44458cce 79957b372307 81a099605936 2024-02-07T12:23:06.397 INFO:teuthology.orchestra.run.smithi078.stdout:mon.a smithi078 running (4m) 23s ago 4m 49.1M 2048M 19.0.0-1252-g44458cce 79957b372307 d62527df0e17 2024-02-07T12:23:06.397 INFO:teuthology.orchestra.run.smithi078.stdout:mon.b smithi106 running (2m) 29s ago 2m 41.7M 2048M 19.0.0-1252-g44458cce 79957b372307 78e5de5d9442 2024-02-07T12:23:06.397 INFO:teuthology.orchestra.run.smithi078.stdout:mon.c smithi171 running (2m) 31s ago 2m 42.9M 2048M 19.0.0-1252-g44458cce 79957b372307 c30ccd326a7c 2024-02-07T12:23:06.397 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 smithi078 running (2m) 23s ago 2m 57.1M 15.6G 19.0.0-1252-g44458cce 79957b372307 6586b51cb5f1 2024-02-07T12:23:06.397 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 smithi106 running (107s) 29s ago 107s 59.4M 15.6G 19.0.0-1252-g44458cce 79957b372307 d41ffd28e7b6 2024-02-07T12:23:06.397 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 smithi171 running (83s) 31s ago 83s 56.9M 16.6G 19.0.0-1252-g44458cce 79957b372307 0c2fe4194f33 2024-02-07T12:23:06.397 INFO:teuthology.orchestra.run.smithi078.stdout:prometheus.a smithi106 *:9095 running (41s) 29s ago 41s 28.0M - 2.43.0 a07b618ecd1d 74e62edb26dc 2024-02-07T12:23:06.397 INFO:teuthology.orchestra.run.smithi078.stdout:rgw.foo.a smithi078 *:80 running (62s) 23s ago 62s 79.2M - 19.0.0-1252-g44458cce 79957b372307 1c13524bc974 2024-02-07T12:23:07.046 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-07T12:23:07.281 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:23:07.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:07 smithi171 ceph-mon[30005]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:07.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:07 smithi171 ceph-mon[30005]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:07.689 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:07 smithi078 ceph-mon[25576]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:07.689 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:07 smithi078 ceph-mon[25576]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:07.689 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:23:07 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:23:07.386Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00395635s 2024-02-07T12:23:07.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:07 smithi106 ceph-mon[29693]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:07.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:07 smithi106 ceph-mon[29693]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:09.074 INFO:teuthology.orchestra.run.smithi078.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-07T12:23:09.074 INFO:teuthology.orchestra.run.smithi078.stdout:alertmanager ?:9093,9094 0/1 - 59s smithi078=a;count:1 2024-02-07T12:23:09.074 INFO:teuthology.orchestra.run.smithi078.stdout:grafana ?:3000 0/1 - 57s smithi106=a;count:1 2024-02-07T12:23:09.074 INFO:teuthology.orchestra.run.smithi078.stdout:mgr 3/3 33s ago 2m smithi078=y;smithi106=x;smithi171=z;count:3 2024-02-07T12:23:09.075 INFO:teuthology.orchestra.run.smithi078.stdout:mon 3/3 33s ago 3m smithi078:172.21.15.78=a;smithi106:172.21.15.106=b;smithi171:172.21.15.171=c;count:3 2024-02-07T12:23:09.075 INFO:teuthology.orchestra.run.smithi078.stdout:node-exporter ?:9100 0/3 - 63s smithi078=a;smithi106=b;smithi171=c;count:3 2024-02-07T12:23:09.075 INFO:teuthology.orchestra.run.smithi078.stdout:osd 3 33s ago - 2024-02-07T12:23:09.075 INFO:teuthology.orchestra.run.smithi078.stdout:prometheus ?:9095 1/1 31s ago 65s smithi106=a;count:1 2024-02-07T12:23:09.075 INFO:teuthology.orchestra.run.smithi078.stdout:rgw.foo.a ?:80 1/1 26s ago 64s smithi078=foo.a;count:1 2024-02-07T12:23:09.648 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:09 smithi106 ceph-mon[29693]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:09.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:09 smithi171 ceph-mon[30005]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:09.673 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:09 smithi078 ceph-mon[25576]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:09.701 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-07T12:23:09.936 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:23:10.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:10 smithi078 ceph-mon[25576]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:10.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:10 smithi171 ceph-mon[30005]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:10.663 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:10 smithi106 ceph-mon[29693]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:11.016 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:10 smithi106 systemd[1]: Starting Ceph grafana.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:11.397 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 podman[42470]: 2024-02-07 12:23:10.953087094 +0000 UTC m=+0.016609163 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-07T12:23:11.397 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 podman[42470]: 2024-02-07 12:23:11.058756802 +0000 UTC m=+0.122278914 container create 517a14cc177120cc0bab575029e930b7ad7b73479a270e2c07e1054fba7be6c4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a, vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , vcs-type=git, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, distribution-scope=public, description=Ceph Grafana Container, name=ubi8, io.openshift.expose-services=) 2024-02-07T12:23:11.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:11 smithi078 ceph-mon[25576]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:11.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-mon[29693]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:11.647 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 podman[42470]: 2024-02-07 12:23:11.443191543 +0000 UTC m=+0.506713630 container init 517a14cc177120cc0bab575029e930b7ad7b73479a270e2c07e1054fba7be6c4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a, name=ubi8, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., version=8.5, distribution-scope=public, maintainer=Paul Cuzner , io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, io.buildah.version=1.32.0, com.redhat.component=ubi8-container) 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 podman[42470]: 2024-02-07 12:23:11.44868658 +0000 UTC m=+0.512208658 container start 517a14cc177120cc0bab575029e930b7ad7b73479a270e2c07e1054fba7be6c4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a, build-date=2022-04-29T11:48:11.055897, vcs-type=git, io.openshift.expose-services=, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, release=239.1651231664, description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public) 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 bash[42470]: 517a14cc177120cc0bab575029e930b7ad7b73479a270e2c07e1054fba7be6c4 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 systemd[1]: Started Ceph grafana.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600284272Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600510011Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600529242Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600537343Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600543427Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600549368Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-07T12:23:11.648 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600554972Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600560708Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.60056656Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600572272Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600577728Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600583048Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.600589227Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=settings t=2024-02-07T12:23:11.60059484Z level=info msg="App mode production" 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=sqlstore t=2024-02-07T12:23:11.60064003Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=sqlstore t=2024-02-07T12:23:11.600658061Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.601510682Z level=info msg="Starting DB migrations" 2024-02-07T12:23:11.649 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.601587939Z level=info msg="Executing migration" id="create migration_log table" 2024-02-07T12:23:11.650 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.602109072Z level=info msg="Executing migration" id="create user table" 2024-02-07T12:23:11.650 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.602547877Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-07T12:23:11.650 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.603030508Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-07T12:23:11.650 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.603500194Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-07T12:23:11.650 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.604034463Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-07T12:23:11.651 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.604433756Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-07T12:23:11.651 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.605151502Z level=info msg="Executing migration" id="create user table v2" 2024-02-07T12:23:11.651 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.605541501Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-07T12:23:11.651 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.605953969Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-07T12:23:11.651 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.606376785Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-07T12:23:11.651 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.606734631Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-07T12:23:11.651 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.607092269Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-07T12:23:11.651 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.607484386Z level=info msg="Executing migration" id="Update user table charset" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.607764399Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.608174842Z level=info msg="Executing migration" id="Add missing user data" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.608504512Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.608939853Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.609345005Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.609749396Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.610832614Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.611275942Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.611724451Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.612114292Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-07T12:23:11.652 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.612516153Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.613466311Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.614317063Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.61693404Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.619773845Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.620334079Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.622988955Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.623701594Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.625982195Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.628791055Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.629267097Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.631871557Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-07T12:23:11.653 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.632316396Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-07T12:23:11.654 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.634484037Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-07T12:23:11.654 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.634864751Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-07T12:23:11.654 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.637427697Z level=info msg="Executing migration" id="create star table" 2024-02-07T12:23:11.654 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.640052099Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-07T12:23:11.654 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.642980354Z level=info msg="Executing migration" id="create org table v1" 2024-02-07T12:23:11.654 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.64563137Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-07T12:23:11.654 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.646057235Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-07T12:23:11.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:11 smithi171 ceph-mon[30005]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:11.737 INFO:teuthology.orchestra.run.smithi078.stdout:HOST ADDR LABELS STATUS 2024-02-07T12:23:11.738 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 172.21.15.78 2024-02-07T12:23:11.738 INFO:teuthology.orchestra.run.smithi078.stdout:smithi106 172.21.15.106 2024-02-07T12:23:11.738 INFO:teuthology.orchestra.run.smithi078.stdout:smithi171 172.21.15.171 2024-02-07T12:23:11.738 INFO:teuthology.orchestra.run.smithi078.stdout:3 hosts in cluster 2024-02-07T12:23:11.899 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.648793623Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-07T12:23:11.899 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.651535013Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.654149465Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.65466432Z level=info msg="Executing migration" id="Update org table charset" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.657285757Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.659964506Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.662691635Z level=info msg="Executing migration" id="create dashboard table" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.665237433Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.667959625Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.670414264Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.673198444Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-07T12:23:11.900 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.675814585Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.678652698Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.681335929Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.683947542Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.686704236Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.689474782Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.692116456Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.694802307Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.695509525Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.696300547Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.69723675Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.697948977Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.698766756Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-07T12:23:11.901 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.699677389Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.700383588Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.701183255Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.701622009Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.701939505Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.702740215Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.703249456Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.704043477Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.704595063Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.704942477Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.705953824Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.70658272Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.707470026Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-07T12:23:11.902 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.708166937Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-07T12:23:11.904 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.709186702Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-07T12:23:11.904 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.710226114Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-07T12:23:11.904 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.711088919Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-07T12:23:11.904 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.713325257Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-07T12:23:11.904 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.715797798Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-07T12:23:11.904 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.718407696Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-07T12:23:11.904 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.719245713Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-07T12:23:11.904 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.719956083Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.720841919Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.721664386Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.722449539Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.723401191Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.724134912Z level=info msg="Executing migration" id="create data_source table" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.724946675Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.725962671Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.726672406Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.727593565Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.728434351Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.729424019Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.730192221Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-07T12:23:11.905 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.73103694Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.731883653Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.732841658Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.733476271Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.734482512Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.735203908Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.736203834Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.736898611Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.737842728Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.738604726Z level=info msg="Executing migration" id="Add uid column" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.739563856Z level=info msg="Executing migration" id="Update uid value" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.740331595Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.741300634Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-07T12:23:11.906 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.741990742Z level=info msg="Executing migration" id="create api_key table" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.742823269Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.743783462Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.744600664Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.745348524Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.746264938Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.746874158Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.747268904Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.750017917Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.752846236Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.75328011Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.75624831Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-07T12:23:11.907 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.758679643Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.761341812Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.761702604Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.764524877Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.767090584Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.769924734Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.770355397Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.773231964Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.776220382Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.778847706Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.779358549Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.782124627Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.78267295Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-07T12:23:11.908 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.784969877Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.787487301Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.787802012Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.790596082Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.793153354Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.795807641Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.796279128Z level=info msg="Executing migration" id="create quota table v1" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.79908159Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.801869766Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.804456573Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.805042884Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.807825094Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.810639829Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-07T12:23:11.909 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.813050487Z level=info msg="Executing migration" id="create session table" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.813538573Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.816316343Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.818945814Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.821826278Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.822282964Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.823149005Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.823671409Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.823978133Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.824897366Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.825433835Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.826317835Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.827217583Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.827556512Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-07T12:23:11.910 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.828449257Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.829090648Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.83000917Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.830837142Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.831606009Z level=info msg="Executing migration" id="create alert table v1" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.832110822Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.833070186Z level=info msg="Executing migration" id="add index alert state" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.833726944Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.834485873Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.834935223Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.83589878Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.836632995Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-07T12:23:11.911 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.838050998Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.838435646Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.839278563Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.840151773Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.841169243Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.84175776Z level=info msg="Executing migration" id="Add column is_default" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.842765501Z level=info msg="Executing migration" id="Add column frequency" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.843537052Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.844381584Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.845072067Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.846009749Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.846778109Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-07T12:23:11.912 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.84779318Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.848308144Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.851152089Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.853959853Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.856685212Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.857183116Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.859795248Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.860622219Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.862405236Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.865326908Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.8658498Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-07T12:23:11.913 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.868746146Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.871420731Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.874310126Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.874834757Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.877509655Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.880253877Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.883031785Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.883456924Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.886440191Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.88903452Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.8917695Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.892215241Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.894651655Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.897224022Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.899779403Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.900320241Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-07T12:23:11.914 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.901098755Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-07T12:23:11.915 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.901871212Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-07T12:23:12.151 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.903869958Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-07T12:23:12.151 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.904249977Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-07T12:23:12.151 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.905078534Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-07T12:23:12.151 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.905864364Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.906642251Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.906964473Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.90777435Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.908571871Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.909410243Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.909879348Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.910653224Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.911449644Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.912260345Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.912567473Z level=info msg="Executing migration" id="Move region to single row" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.913347892Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-07T12:23:12.152 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.914168441Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-07T12:23:12.153 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.915005376Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-07T12:23:12.153 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.915451352Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.91626161Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.917064902Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.917959156Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.918322571Z level=info msg="Executing migration" id="create test_data table" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.921202031Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.921788629Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.924749122Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.925210474Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.927882963Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-07T12:23:12.154 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.928273164Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.930634095Z level=info msg="Executing migration" id="create team table" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.931196852Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.933502314Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.93395133Z level=info msg="Executing migration" id="create team member table" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.936631249Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.93932864Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.942075768Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.942528611Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.945545049Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.948209207Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-07T12:23:12.155 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.951094407Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-07T12:23:12.156 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.951528699Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-07T12:23:12.156 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.95417314Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-07T12:23:12.156 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.956859951Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.957321728Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.957798231Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.960257407Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.960726143Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.961168865Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.96152929Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.961791181Z level=info msg="Executing migration" id="create tag table" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.962212025Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.962637912Z level=info msg="Executing migration" id="create login attempt table" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.963042704Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.963467223Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.963874158Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.965678877Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-07T12:23:12.157 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.966089726Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.966515934Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.966796621Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.967171351Z level=info msg="Executing migration" id="create user auth table" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.967517544Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.967939399Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.968222152Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.968940016Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.969647328Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.970392252Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.971181046Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.971592926Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.972368026Z level=info msg="Executing migration" id="create server_lock table" 2024-02-07T12:23:12.158 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.972779002Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.973233969Z level=info msg="Executing migration" id="create user auth token table" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.973658859Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.97629424Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.976769553Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.977215008Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.977982503Z level=info msg="Executing migration" id="create cache_data table" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.978398826Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.978830542Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.979286573Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.979727264Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.980012952Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-07T12:23:12.159 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.980277222Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.982051827Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.984525231Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.987055936Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.989597392Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.990054243Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.992854732Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.995583904Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:11 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:11.998445914Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.001027567Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.003749697Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.006309695Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-07T12:23:12.160 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.009005787Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.011728288Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.014354229Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.015149463Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.015965933Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.016867473Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.01754059Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.018450808Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.019342875Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.020124259Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.021033218Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.024435799Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.027664812Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-07T12:23:12.161 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.028488164Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.029394149Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.030394783Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.030844756Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.031615965Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.032397366Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.033209119Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.033973967Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.034810904Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.035635017Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.036479799Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.03729978Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.038238687Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.039176462Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-07T12:23:12.162 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.040034392Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.040976423Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.041939236Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.042775513Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.043567519Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.044384798Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.045334863Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.0459659Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.046259073Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.047141656Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.048016833Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.048878818Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.049736694Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.052517507Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.055297245Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-07T12:23:12.163 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.057871116Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.060439142Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.063154495Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.065807042Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.068443986Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.071133985Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.073801122Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.076470412Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.079227886Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.081939509Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.084580014Z level=info msg="Executing migration" id="create alert_image table" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.087440862Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-07T12:23:12.164 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.089928484Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-07T12:23:12.165 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.09264025Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-07T12:23:12.165 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.093494803Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-07T12:23:12.165 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.094257495Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-07T12:23:12.165 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.094349419Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-07T12:23:12.165 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.095209769Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.095970785Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.096753948Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.096867569Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.096973017Z level=info msg="alerts found to migrate" alerts=0 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.097574307Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.098539668Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.099255606Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.100039799Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.10084975Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.101748365Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.102654014Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-07T12:23:12.166 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.103406145Z level=info msg="Executing migration" id="create data_keys table" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.104236199Z level=info msg="Executing migration" id="create secrets table" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.105068198Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.109339028Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.110286601Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.111091666Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.115309543Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.119330927Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.120178568Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.12119331Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.12190891Z level=info msg="Executing migration" id="create permission table" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.122786015Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.123621738Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.124648864Z level=info msg="Executing migration" id="create role table" 2024-02-07T12:23:12.167 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.125324509Z level=info msg="Executing migration" id="add column display_name" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.126268502Z level=info msg="Executing migration" id="add column group_name" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.127202178Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.128084108Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.128883636Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.129704992Z level=info msg="Executing migration" id="create team role table" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.130560623Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.131535482Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.132220016Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.135130072Z level=info msg="Executing migration" id="create user role table" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.137780514Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.140563045Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.143507346Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.143957947Z level=info msg="Executing migration" id="create builtin role table" 2024-02-07T12:23:12.168 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.146670428Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-07T12:23:12.169 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.149575942Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-07T12:23:12.169 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.152154264Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-07T12:23:12.169 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.153207994Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-07T12:23:12.324 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:12 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ::ffff:172.21.15.106 - - [07/Feb/2024:12:23:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T12:23:12.348 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-07T12:23:12.516 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.155947527Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-07T12:23:12.516 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.158727116Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.161561423Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.162020992Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.164803158Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.167644745Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.170408184Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.17091977Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.173464186Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.17595044Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.17857171Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.179067321Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.18179919Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-07T12:23:12.517 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.184544121Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-07T12:23:12.518 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.187107402Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-07T12:23:12.518 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.187515214Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-07T12:23:12.518 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.190233085Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-07T12:23:12.518 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.192744133Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-07T12:23:12.518 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.19560883Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.198190429Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.201033399Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.2034026Z level=info msg="Executing migration" id="add correlation config column" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.206184715Z level=info msg="Executing migration" id="create entity_events table" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.206700268Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.20943417Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.209839589Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.21196659Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.212057981Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.212776364Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.213195859Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.214095247Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-07T12:23:12.519 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.214655536Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.215116567Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.215934039Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.216369772Z level=info msg="Executing migration" id="Drop public config table" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.217194121Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.217618721Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.218079615Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.218942451Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.21977336Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.222770141Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.223774334Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-07T12:23:12.520 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.224769262Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.22509202Z level=info msg="Executing migration" id="add share column" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.226078373Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.226931044Z level=info msg="Executing migration" id="create file table" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.227855505Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.228287592Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.229244348Z level=info msg="Executing migration" id="create file_meta table" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.230030065Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.230897562Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.231190597Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.232047642Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.232377092Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.232896788Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-07T12:23:12.521 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.233903639Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.234214431Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.235156257Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.235933846Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.236777952Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.237746473Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.238983004Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.240001284Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.242915052Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.252550981Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.253019397Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-07T12:23:12.522 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.256053741Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.259602969Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.262556754Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.263057292Z level=info msg="Executing migration" id="create folder table" 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.265811914Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.266302274Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.268840465Z level=info msg="Executing migration" id="Update folder title length" 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.27173401Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=migrator t=2024-02-07T12:23:12.272367278Z level=info msg="migrations completed" performed=485 skipped=0 duration=670.802313ms 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=sqlstore t=2024-02-07T12:23:12.272753769Z level=info msg="Created default organization" 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=plugin.loader t=2024-02-07T12:23:12.330844699Z level=info msg="Plugin registered" pluginID=input 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=plugin.loader t=2024-02-07T12:23:12.357178621Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=plugin.loader t=2024-02-07T12:23:12.357198041Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=secrets t=2024-02-07T12:23:12.35722958Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-07T12:23:12.523 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=query_data t=2024-02-07T12:23:12.359233435Z level=info msg="Query Service initialization" 2024-02-07T12:23:12.524 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=live.push_http t=2024-02-07T12:23:12.361690362Z level=info msg="Live Push Gateway initialization" 2024-02-07T12:23:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:23:12.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:12 smithi078 ceph-mon[25576]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:12.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:12 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:12 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:12 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:12 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:12 smithi078 ceph-mon[25576]: Deploying daemon node-exporter.a on smithi078 2024-02-07T12:23:12.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:12 smithi078 ceph-mon[25576]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:12.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:12 smithi171 ceph-mon[30005]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:12.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:12 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:12 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:12 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:12 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:12 smithi171 ceph-mon[30005]: Deploying daemon node-exporter.a on smithi078 2024-02-07T12:23:12.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:12 smithi171 ceph-mon[30005]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:12.968 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-mon[29693]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:12.968 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.968 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.968 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.968 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:12.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-mon[29693]: Deploying daemon node-exporter.a on smithi078 2024-02-07T12:23:12.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-mon[29693]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:13.266 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=infra.usagestats.collector t=2024-02-07T12:23:12.966843138Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-07T12:23:13.266 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=provisioning.datasources t=2024-02-07T12:23:12.967824377Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-07T12:23:13.266 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=provisioning.datasources t=2024-02-07T12:23:12.978708859Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-07T12:23:13.266 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=provisioning.alerting t=2024-02-07T12:23:12.989420203Z level=info msg="starting to provision alerting" 2024-02-07T12:23:13.266 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=provisioning.alerting t=2024-02-07T12:23:12.98943793Z level=info msg="finished to provision alerting" 2024-02-07T12:23:13.267 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=ngalert.state.manager t=2024-02-07T12:23:12.989503252Z level=info msg="Warming state cache for startup" 2024-02-07T12:23:13.267 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=grafanaStorageLogger t=2024-02-07T12:23:12.989636829Z level=info msg="storage starting" 2024-02-07T12:23:13.267 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=ngalert.state.manager t=2024-02-07T12:23:12.989661612Z level=info msg="State cache has been initialized" states=0 duration=157.489µs 2024-02-07T12:23:13.267 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=ticker t=2024-02-07T12:23:12.989736587Z level=info msg=starting first_tick=2024-02-07T12:23:20Z 2024-02-07T12:23:13.267 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=ngalert.multiorg.alertmanager t=2024-02-07T12:23:12.989760264Z level=info msg="starting MultiOrg Alertmanager" 2024-02-07T12:23:13.267 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:12 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=http.server t=2024-02-07T12:23:12.996936316Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-07T12:23:13.267 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:13 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=sqlstore.transactions t=2024-02-07T12:23:13.00301239Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-07T12:23:13.267 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:23:13 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a[42485]: logger=sqlstore.transactions t=2024-02-07T12:23:13.015413832Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-07T12:23:14.608 INFO:teuthology.orchestra.run.smithi078.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-07T12:23:14.608 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-07T12:23:14.608 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme1n1 ssd Linux_87964fc57034fcb8e951 89.4G Yes 2m ago 2024-02-07T12:23:14.608 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme2n1 ssd Linux_9f094c66bb0f6918158b 89.4G Yes 2m ago 2024-02-07T12:23:14.608 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme3n1 ssd Linux_db7cd0359163b82d7693 89.4G Yes 2m ago 2024-02-07T12:23:14.608 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme4n1 ssd Linux_a99c2fd380d19918a907 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-07T12:23:14.608 INFO:teuthology.orchestra.run.smithi078.stdout:smithi106 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013R400BGN 372G No 108s ago Has a FileSystem, LVM detected 2024-02-07T12:23:14.609 INFO:teuthology.orchestra.run.smithi078.stdout:smithi106 /dev/nvme1n1 ssd Linux_dc0d812a6363dc70dd80 89.4G Yes 108s ago 2024-02-07T12:23:14.609 INFO:teuthology.orchestra.run.smithi078.stdout:smithi106 /dev/nvme2n1 ssd Linux_a746df0f88e09fa5eb0f 89.4G Yes 108s ago 2024-02-07T12:23:14.609 INFO:teuthology.orchestra.run.smithi078.stdout:smithi106 /dev/nvme3n1 ssd Linux_353d0ef6347aaf7ebc45 89.4G Yes 108s ago 2024-02-07T12:23:14.609 INFO:teuthology.orchestra.run.smithi078.stdout:smithi106 /dev/nvme4n1 ssd Linux_186f5d0517d4df19c475 89.4G No 108s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-07T12:23:14.609 INFO:teuthology.orchestra.run.smithi078.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 84s ago Has a FileSystem, LVM detected 2024-02-07T12:23:14.609 INFO:teuthology.orchestra.run.smithi078.stdout:smithi171 /dev/nvme1n1 ssd Linux_ad5d6d85b611333c766c 89.4G Yes 84s ago 2024-02-07T12:23:14.609 INFO:teuthology.orchestra.run.smithi078.stdout:smithi171 /dev/nvme2n1 ssd Linux_3736b4043a6b92f3e579 89.4G Yes 84s ago 2024-02-07T12:23:14.609 INFO:teuthology.orchestra.run.smithi078.stdout:smithi171 /dev/nvme3n1 ssd Linux_bd6d59cceaa621335d02 89.4G Yes 84s ago 2024-02-07T12:23:14.609 INFO:teuthology.orchestra.run.smithi078.stdout:smithi171 /dev/nvme4n1 ssd Linux_a8bc866201e75415c106 89.4G No 84s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-07T12:23:14.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:14 smithi078 ceph-mon[25576]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:14.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:14 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:14.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:14 smithi171 ceph-mon[30005]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:14.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:14 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:15.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:14 smithi106 ceph-mon[29693]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:15.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:14 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:15.792 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls --format yaml' 2024-02-07T12:23:15.817 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:15 smithi078 ceph-mon[25576]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:15.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:15 smithi171 ceph-mon[30005]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:16.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:15 smithi106 ceph-mon[29693]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:23:16.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:16 smithi078 ceph-mon[25576]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:16.881 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:16 smithi078 bash[55171]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-07T12:23:16.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:16 smithi171 ceph-mon[30005]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:17.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:16 smithi106 ceph-mon[29693]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:17.339 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:17 smithi078 bash[55171]: Writing manifest to image destination 2024-02-07T12:23:17.631 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:17 smithi078 podman[55171]: 2024-02-07 12:23:17.318584909 +0000 UTC m=+3.408911924 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-07T12:23:18.131 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:17 smithi078 podman[55171]: 2024-02-07 12:23:17.835538623 +0000 UTC m=+3.925865633 container create fd764afa2f688969a248925b70438ad17e5e17c7af3bd80ce4376089acc56e3a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T12:23:18.460 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 podman[55171]: 2024-02-07 12:23:18.265046029 +0000 UTC m=+4.355373052 container init fd764afa2f688969a248925b70438ad17e5e17c7af3bd80ce4376089acc56e3a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T12:23:18.460 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 podman[55171]: 2024-02-07 12:23:18.270347185 +0000 UTC m=+4.360674195 container start fd764afa2f688969a248925b70438ad17e5e17c7af3bd80ce4376089acc56e3a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T12:23:18.460 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.274Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-07T12:23:18.460 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.274Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.274Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.274Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.274Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.274Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=arp 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-07T12:23:18.461 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=edac 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-07T12:23:18.462 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=os 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=stat 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=time 2024-02-07T12:23:18.463 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-07T12:23:18.464 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=uname 2024-02-07T12:23:18.464 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-07T12:23:18.464 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-07T12:23:18.464 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-07T12:23:18.464 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-07T12:23:18.464 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a[55447]: ts=2024-02-07T12:23:18.275Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-07T12:23:18.464 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 bash[55171]: fd764afa2f688969a248925b70438ad17e5e17c7af3bd80ce4376089acc56e3a 2024-02-07T12:23:18.721 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:23:18 smithi078 systemd[1]: Started Ceph node-exporter.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:23:19.059 INFO:teuthology.orchestra.run.smithi078.stdout:service_type: alertmanager 2024-02-07T12:23:19.059 INFO:teuthology.orchestra.run.smithi078.stdout:service_name: alertmanager 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout:placement: 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout: count: 1 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout: hosts: 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi078=a 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout:status: 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout: created: '2024-02-07T12:22:09.073364Z' 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout: ports: 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout: - 9093 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout: - 9094 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout: running: 0 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout: size: 1 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout:events: 2024-02-07T12:23:19.060 INFO:teuthology.orchestra.run.smithi078.stdout:- 2024-02-07T12:22:57.562191Z service:alertmanager [INFO] "service was created" 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout:--- 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout:service_type: grafana 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout:service_name: grafana 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout:placement: 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout: count: 1 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout: hosts: 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi106=a 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout:spec: 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout: anonymous_access: true 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout: protocol: https 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout:status: 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout: created: '2024-02-07T12:22:11.814151Z' 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout: ports: 2024-02-07T12:23:19.061 INFO:teuthology.orchestra.run.smithi078.stdout: - 3000 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout: running: 0 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout: size: 1 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout:events: 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout:- 2024-02-07T12:23:11.656142Z service:grafana [INFO] "service was created" 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout:--- 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout:service_type: mgr 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout:service_name: mgr 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout:placement: 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout: count: 3 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout: hosts: 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi078=y 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi106=x 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi171=z 2024-02-07T12:23:19.062 INFO:teuthology.orchestra.run.smithi078.stdout:status: 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout: created: '2024-02-07T12:20:31.597169Z' 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout: last_refresh: '2024-02-07T12:22:35.256646Z' 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout: running: 3 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout: size: 3 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout:--- 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout:service_type: mon 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout:service_name: mon 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout:placement: 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout: count: 3 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout: hosts: 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi078:172.21.15.78=a 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi106:172.21.15.106=b 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi171:172.21.15.171=c 2024-02-07T12:23:19.063 INFO:teuthology.orchestra.run.smithi078.stdout:status: 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout: created: '2024-02-07T12:20:06.094695Z' 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout: last_refresh: '2024-02-07T12:22:35.256527Z' 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout: running: 3 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout: size: 3 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout:--- 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout:service_type: node-exporter 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout:service_name: node-exporter 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout:placement: 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout: count: 3 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout: hosts: 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi078=a 2024-02-07T12:23:19.064 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi106=b 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi171=c 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:status: 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout: created: '2024-02-07T12:22:05.979226Z' 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout: ports: 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout: - 9100 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout: running: 0 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout: size: 3 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:events: 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:- 2024-02-07T12:23:18.538469Z service:node-exporter [INFO] "service was created" 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:--- 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:service_type: osd 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:service_name: osd 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:unmanaged: true 2024-02-07T12:23:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:spec: 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: filter_logic: AND 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: objectstore: bluestore 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout:status: 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: container_image_id: 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: last_refresh: '2024-02-07T12:22:35.256723Z' 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: running: 3 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: size: 3 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout:--- 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout:service_type: prometheus 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout:service_name: prometheus 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout:placement: 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: count: 1 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: hosts: 2024-02-07T12:23:19.066 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi106=a 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout:status: 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout: created: '2024-02-07T12:22:03.413965Z' 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout: last_refresh: '2024-02-07T12:22:37.110009Z' 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout: ports: 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout: - 9095 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout: running: 1 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout: size: 1 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout:--- 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout:service_type: rgw 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout:service_id: foo.a 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout:service_name: rgw.foo.a 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout:placement: 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout: count: 1 2024-02-07T12:23:19.067 INFO:teuthology.orchestra.run.smithi078.stdout: hosts: 2024-02-07T12:23:19.068 INFO:teuthology.orchestra.run.smithi078.stdout: - smithi078=foo.a 2024-02-07T12:23:19.068 INFO:teuthology.orchestra.run.smithi078.stdout:status: 2024-02-07T12:23:19.068 INFO:teuthology.orchestra.run.smithi078.stdout: created: '2024-02-07T12:22:04.263964Z' 2024-02-07T12:23:19.068 INFO:teuthology.orchestra.run.smithi078.stdout: last_refresh: '2024-02-07T12:22:42.720423Z' 2024-02-07T12:23:19.068 INFO:teuthology.orchestra.run.smithi078.stdout: ports: 2024-02-07T12:23:19.068 INFO:teuthology.orchestra.run.smithi078.stdout: - 80 2024-02-07T12:23:19.068 INFO:teuthology.orchestra.run.smithi078.stdout: running: 1 2024-02-07T12:23:19.068 INFO:teuthology.orchestra.run.smithi078.stdout: size: 1 2024-02-07T12:23:19.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:19 smithi078 ceph-mon[25576]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:19.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:19 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:19.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:19 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:19.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:19 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:19.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:19 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:23:19.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:19 smithi171 ceph-mon[30005]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:19.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:19 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:19.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:19 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:19.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:19 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:19.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:19 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:23:19.683 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:19 smithi106 ceph-mon[29693]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:19.683 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:19 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:19.683 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:19 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:19.683 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:19 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:19.683 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:19 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T12:23:19.683 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:19 smithi106 systemd[1]: Starting Ceph node-exporter.b for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:20.006 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-02-07T12:23:20.015 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:19 smithi106 bash[42760]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-07T12:23:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T12:23:20.389 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:20 smithi106 bash[42760]: Getting image source signatures 2024-02-07T12:23:20.389 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:20 smithi106 bash[42760]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-07T12:23:20.390 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:20 smithi106 bash[42760]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-07T12:23:20.390 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:20 smithi106 bash[42760]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-07T12:23:20.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:20 smithi078 ceph-mon[25576]: Deploying daemon node-exporter.b on smithi106 2024-02-07T12:23:20.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:20 smithi078 ceph-mon[25576]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-07T12:23:20.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:20 smithi171 ceph-mon[30005]: Deploying daemon node-exporter.b on smithi106 2024-02-07T12:23:20.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:20 smithi171 ceph-mon[30005]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-07T12:23:20.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:20 smithi106 ceph-mon[29693]: Deploying daemon node-exporter.b on smithi106 2024-02-07T12:23:20.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:20 smithi106 ceph-mon[29693]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-07T12:23:21.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:21 smithi106 ceph-mon[29693]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:21.516 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:21 smithi106 bash[42760]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-07T12:23:21.516 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:21 smithi106 bash[42760]: Writing manifest to image destination 2024-02-07T12:23:21.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:21 smithi078 ceph-mon[25576]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:21.661 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:21 smithi171 ceph-mon[30005]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:21 smithi106 podman[42760]: 2024-02-07 12:23:21.815011419 +0000 UTC m=+2.225678621 container create bbf751681d39d7f9c5ff2bb83add51fde8e8aab2bdfd63d45ce38103443770ba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:21 smithi106 podman[42760]: 2024-02-07 12:23:21.724630513 +0000 UTC m=+2.135297720 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 podman[42760]: 2024-02-07 12:23:22.025098951 +0000 UTC m=+2.435766164 container init bbf751681d39d7f9c5ff2bb83add51fde8e8aab2bdfd63d45ce38103443770ba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 podman[42760]: 2024-02-07 12:23:22.030692133 +0000 UTC m=+2.441359340 container start bbf751681d39d7f9c5ff2bb83add51fde8e8aab2bdfd63d45ce38103443770ba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.034Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.034Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.034Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.034Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.034Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.034Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-07T12:23:22.119 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-07T12:23:22.120 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=arp 2024-02-07T12:23:22.120 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-07T12:23:22.120 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-07T12:23:22.120 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-07T12:23:22.120 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-07T12:23:22.120 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-07T12:23:22.120 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=edac 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-07T12:23:22.121 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=os 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-07T12:23:22.122 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=stat 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=time 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=uname 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b[42810]: ts=2024-02-07T12:23:22.035Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-07T12:23:22.123 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 bash[42760]: bbf751681d39d7f9c5ff2bb83add51fde8e8aab2bdfd63d45ce38103443770ba 2024-02-07T12:23:22.207 INFO:teuthology.orchestra.run.smithi078.stdout:osd 3 46s ago - 2024-02-07T12:23:22.381 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:22 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: ::ffff:172.21.15.106 - - [07/Feb/2024:12:23:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T12:23:22.515 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:23:22 smithi106 systemd[1]: Started Ceph node-exporter.b for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:23:22.910 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-07T12:23:22.920 INFO:tasks.cephadm:Teardown begin 2024-02-07T12:23:22.921 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T12:23:22.950 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T12:23:22.983 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T12:23:23.142 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-07T12:23:23.142 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-07T12:23:23.157 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-07T12:23:23.175 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-07T12:23:23.191 INFO:tasks.cephadm:Stopping all daemons... 2024-02-07T12:23:23.191 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-07T12:23:23.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.a 2024-02-07T12:23:23.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:23 smithi106 ceph-mon[29693]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:23.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:23 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:23.265 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:23 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:23.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:23 smithi106 ceph-mon[29693]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:23.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:23 smithi106 ceph-mon[29693]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:23.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:23 smithi106 ceph-mon[29693]: Deploying daemon node-exporter.c on smithi171 2024-02-07T12:23:23.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:23 smithi171 ceph-mon[30005]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:23.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:23 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:23.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:23 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:23.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:23 smithi171 ceph-mon[30005]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:23.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:23 smithi171 ceph-mon[30005]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:23.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:23 smithi171 ceph-mon[30005]: Deploying daemon node-exporter.c on smithi171 2024-02-07T12:23:23.413 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:23 smithi171 bash[43532]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-07T12:23:23.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-mon[25576]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T12:23:23.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:23.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:23.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-mon[25576]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T12:23:23.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-mon[25576]: from='mgr.14418 172.21.15.78:0/1092423574' entity='mgr.y' 2024-02-07T12:23:23.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-mon[25576]: Deploying daemon node-exporter.c on smithi171 2024-02-07T12:23:23.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 systemd[1]: Stopping Ceph mon.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:23.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:23:23] ENGINE Bus STOPPING 2024-02-07T12:23:23.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:23:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-07T12:23:23.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:23:23] ENGINE Bus STOPPED 2024-02-07T12:23:23.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:23:23] ENGINE Bus STARTING 2024-02-07T12:23:23.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:23:23] ENGINE Serving on http://:::9283 2024-02-07T12:23:23.881 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: [07/Feb/2024:12:23:23] ENGINE Bus STARTED 2024-02-07T12:23:23.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a[25553]: 2024-02-07T12:23:23.558+0000 7fca4a44e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T12:23:23.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a[25553]: 2024-02-07T12:23:23.558+0000 7fca4a44e700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-07T12:23:23.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:23 smithi078 podman[55850]: 2024-02-07 12:23:23.630756627 +0000 UTC m=+0.252680924 container died d62527df0e17bc417459490415bc0199e777110253aa043c14409e2175b457e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-07T12:23:24.162 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:23 smithi171 bash[43532]: Getting image source signatures 2024-02-07T12:23:24.162 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:23 smithi171 bash[43532]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-07T12:23:24.162 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:23 smithi171 bash[43532]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-07T12:23:24.162 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:23 smithi171 bash[43532]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-07T12:23:24.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:24 smithi078 podman[55850]: 2024-02-07 12:23:24.410803884 +0000 UTC m=+1.032728217 container cleanup d62527df0e17bc417459490415bc0199e777110253aa043c14409e2175b457e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T12:23:24.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Feb 07 12:23:24 smithi078 bash[55850]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-a 2024-02-07T12:23:24.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.a.service' 2024-02-07T12:23:24.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:23:24.853 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-07T12:23:24.853 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-07T12:23:24.853 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.b 2024-02-07T12:23:25.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:24 smithi106 systemd[1]: Stopping Ceph mon.b for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:25.412 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 bash[43532]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-07T12:23:25.412 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 bash[43532]: Writing manifest to image destination 2024-02-07T12:23:25.480 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-b[29670]: 2024-02-07T12:23:25.172+0000 7f29b195c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T12:23:25.481 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:25 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-b[29670]: 2024-02-07T12:23:25.172+0000 7f29b195c700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-07T12:23:25.481 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:25 smithi106 podman[42923]: 2024-02-07 12:23:25.234420466 +0000 UTC m=+0.192718726 container died 78e5de5d944227790361f61596cf3f23f26ad130f31612d06a68b87ffd0e8b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T12:23:25.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:25 smithi106 podman[42923]: 2024-02-07 12:23:25.569965108 +0000 UTC m=+0.528263367 container cleanup 78e5de5d944227790361f61596cf3f23f26ad130f31612d06a68b87ffd0e8b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-b, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-07T12:23:25.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:25 smithi106 bash[42923]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-b 2024-02-07T12:23:25.846 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 podman[43532]: 2024-02-07 12:23:25.518975584 +0000 UTC m=+2.259385765 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-07T12:23:25.846 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 podman[43532]: 2024-02-07 12:23:25.600607725 +0000 UTC m=+2.341017905 container create 54cc3d85a17b7d65fb0deb5bb022589a6745b2ce1d7cbf45f7b49027efd6191b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T12:23:25.846 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 podman[43532]: 2024-02-07 12:23:25.844087916 +0000 UTC m=+2.584498096 container init 54cc3d85a17b7d65fb0deb5bb022589a6745b2ce1d7cbf45f7b49027efd6191b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T12:23:26.031 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.b.service' 2024-02-07T12:23:26.054 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:25 smithi106 podman[42942]: 2024-02-07 12:23:25.796102182 +0000 UTC m=+0.565459857 container remove 78e5de5d944227790361f61596cf3f23f26ad130f31612d06a68b87ffd0e8b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-b, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-07T12:23:26.054 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:26 smithi106 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.b.service: Deactivated successfully. 2024-02-07T12:23:26.054 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:26 smithi106 systemd[1]: Stopped Ceph mon.b for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:23:26.054 INFO:journalctl@ceph.mon.b.smithi106.stdout:Feb 07 12:23:26 smithi106 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.b.service: Consumed 3.463s CPU time. 2024-02-07T12:23:26.076 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:23:26.076 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-07T12:23:26.076 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-07T12:23:26.077 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.c 2024-02-07T12:23:26.105 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 podman[43532]: 2024-02-07 12:23:25.850428078 +0000 UTC m=+2.590838259 container start 54cc3d85a17b7d65fb0deb5bb022589a6745b2ce1d7cbf45f7b49027efd6191b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T12:23:26.105 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.855Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-07T12:23:26.105 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.855Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-07T12:23:26.105 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=arp 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-07T12:23:26.106 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-07T12:23:26.107 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=edac 2024-02-07T12:23:26.107 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-07T12:23:26.107 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-07T12:23:26.107 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-07T12:23:26.107 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-07T12:23:26.107 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-07T12:23:26.108 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=os 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=stat 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=time 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-07T12:23:26.109 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=uname 2024-02-07T12:23:26.110 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-07T12:23:26.110 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-07T12:23:26.110 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.856Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-07T12:23:26.110 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.857Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-07T12:23:26.110 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c[43583]: ts=2024-02-07T12:23:25.857Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-07T12:23:26.110 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 bash[43532]: 54cc3d85a17b7d65fb0deb5bb022589a6745b2ce1d7cbf45f7b49027efd6191b 2024-02-07T12:23:26.110 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:23:25 smithi171 systemd[1]: Started Ceph node-exporter.c for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:23:26.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:26 smithi171 systemd[1]: Stopping Ceph mon.c for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:26.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:26 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-c[29982]: 2024-02-07T12:23:26.382+0000 7f9683d6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T12:23:26.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:26 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-c[29982]: 2024-02-07T12:23:26.382+0000 7f9683d6d700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-07T12:23:26.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:26 smithi171 podman[43662]: 2024-02-07 12:23:26.556840994 +0000 UTC m=+0.284659441 container died c30ccd326a7c82ed6155ff4d4dcec8c896bb356789c27e2e9b3ae37f17c41fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-c, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-07T12:23:27.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:27 smithi171 podman[43662]: 2024-02-07 12:23:27.021790435 +0000 UTC m=+0.749608929 container cleanup c30ccd326a7c82ed6155ff4d4dcec8c896bb356789c27e2e9b3ae37f17c41fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131) 2024-02-07T12:23:27.162 INFO:journalctl@ceph.mon.c.smithi171.stdout:Feb 07 12:23:27 smithi171 bash[43662]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mon-c 2024-02-07T12:23:27.399 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mon.c.service' 2024-02-07T12:23:27.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:23:27.486 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-07T12:23:27.487 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-07T12:23:27.487 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.y 2024-02-07T12:23:27.778 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:27 smithi078 systemd[1]: Stopping Ceph mgr.y for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:28.076 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:27 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y[25830]: 2024-02-07T12:23:27.775+0000 7f33b8666700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T12:23:28.076 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:27 smithi078 podman[55991]: 2024-02-07 12:23:27.807966091 +0000 UTC m=+0.129302655 container died fa19eb12e0b53e5d624f4103e63f0d8632575c88b7b7d4f462bcf93d80d6c83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T12:23:28.381 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:28 smithi078 podman[55991]: 2024-02-07 12:23:28.223735819 +0000 UTC m=+0.545072423 container cleanup fa19eb12e0b53e5d624f4103e63f0d8632575c88b7b7d4f462bcf93d80d6c83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T12:23:28.381 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Feb 07 12:23:28 smithi078 bash[55991]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-y 2024-02-07T12:23:28.650 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.y.service' 2024-02-07T12:23:28.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:23:28.687 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-07T12:23:28.687 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-07T12:23:28.687 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.x 2024-02-07T12:23:28.999 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:23:28 smithi106 systemd[1]: Stopping Ceph mgr.x for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:29.265 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:23:28 smithi106 podman[43056]: 2024-02-07 12:23:28.997659936 +0000 UTC m=+0.120866644 container died 3deecab659e4a0bba5b9c1afa811be0b44abd670736b15ef42cac9c42bf624ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T12:23:29.643 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:23:29 smithi106 podman[43056]: 2024-02-07 12:23:29.373345055 +0000 UTC m=+0.496551744 container cleanup 3deecab659e4a0bba5b9c1afa811be0b44abd670736b15ef42cac9c42bf624ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T12:23:29.643 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Feb 07 12:23:29 smithi106 bash[43056]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-x 2024-02-07T12:23:29.788 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.x.service' 2024-02-07T12:23:29.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:23:29.825 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-07T12:23:29.825 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-07T12:23:29.825 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.z 2024-02-07T12:23:30.161 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:23:29 smithi171 systemd[1]: Stopping Ceph mgr.z for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:30.412 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:23:30 smithi171 podman[43794]: 2024-02-07 12:23:30.159763297 +0000 UTC m=+0.147470149 container died 81a09960593613277f21a9f1e4e780803328659335936c16e55bdf9de27e94a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T12:23:30.804 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:23:30 smithi171 podman[43794]: 2024-02-07 12:23:30.518709369 +0000 UTC m=+0.506416227 container cleanup 81a09960593613277f21a9f1e4e780803328659335936c16e55bdf9de27e94a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-07T12:23:30.804 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Feb 07 12:23:30 smithi171 bash[43794]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-mgr-z 2024-02-07T12:23:30.944 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@mgr.z.service' 2024-02-07T12:23:30.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:23:30.979 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-07T12:23:30.979 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-07T12:23:30.979 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@osd.0 2024-02-07T12:23:31.282 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:31 smithi078 systemd[1]: Stopping Ceph osd.0 for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:31.631 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:31 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0[40083]: 2024-02-07T12:23:31.279+0000 7f121e334700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T12:23:31.631 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:31 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0[40083]: 2024-02-07T12:23:31.279+0000 7f121e334700 -1 osd.0 31 *** Got signal Terminated *** 2024-02-07T12:23:31.631 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:31 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0[40083]: 2024-02-07T12:23:31.279+0000 7f121e334700 -1 osd.0 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-07T12:23:37.047 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:36 smithi078 podman[56121]: 2024-02-07 12:23:36.747533351 +0000 UTC m=+5.581173376 container died 6586b51cb5f18067e7238b188f8f30dd77590b22a5aad96734d7dc0fee373741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-07T12:23:37.364 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:37 smithi078 podman[56121]: 2024-02-07 12:23:37.202814527 +0000 UTC m=+6.036454553 container cleanup 6586b51cb5f18067e7238b188f8f30dd77590b22a5aad96734d7dc0fee373741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.2) 2024-02-07T12:23:37.364 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:37 smithi078 bash[56121]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0 2024-02-07T12:23:37.631 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:37 smithi078 podman[56142]: 2024-02-07 12:23:37.362530557 +0000 UTC m=+0.613970890 container remove 6586b51cb5f18067e7238b188f8f30dd77590b22a5aad96734d7dc0fee373741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-07T12:23:38.023 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:37 smithi078 podman[56232]: 2024-02-07 12:23:37.669139361 +0000 UTC m=+0.017334785 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T12:23:38.023 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:37 smithi078 podman[56232]: 2024-02-07 12:23:37.77152297 +0000 UTC m=+0.119718389 container create 38337703cf9ff0f972889fa6774a77befacb39f9b0646096d9fcb3d52979dddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-07T12:23:38.023 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:38 smithi078 podman[56232]: 2024-02-07 12:23:38.015605494 +0000 UTC m=+0.363800912 container init 38337703cf9ff0f972889fa6774a77befacb39f9b0646096d9fcb3d52979dddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-07T12:23:38.023 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:38 smithi078 podman[56232]: 2024-02-07 12:23:38.021109295 +0000 UTC m=+0.369304716 container start 38337703cf9ff0f972889fa6774a77befacb39f9b0646096d9fcb3d52979dddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2) 2024-02-07T12:23:38.308 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:38 smithi078 podman[56232]: 2024-02-07 12:23:38.132426916 +0000 UTC m=+0.480622339 container attach 38337703cf9ff0f972889fa6774a77befacb39f9b0646096d9fcb3d52979dddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T12:23:38.631 INFO:journalctl@ceph.osd.0.smithi078.stdout:Feb 07 12:23:38 smithi078 podman[56232]: 2024-02-07 12:23:38.307826237 +0000 UTC m=+0.656021656 container died 38337703cf9ff0f972889fa6774a77befacb39f9b0646096d9fcb3d52979dddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-0-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-07T12:23:38.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@osd.0.service' 2024-02-07T12:23:38.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:23:38.981 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-07T12:23:38.982 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-07T12:23:38.982 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@osd.1 2024-02-07T12:23:39.265 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:39 smithi106 systemd[1]: Stopping Ceph osd.1 for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:40.516 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:40 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1[34241]: 2024-02-07T12:23:40.188+0000 7ffb8e7d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T12:23:40.516 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:40 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1[34241]: 2024-02-07T12:23:40.188+0000 7ffb8e7d8700 -1 osd.1 31 *** Got signal Terminated *** 2024-02-07T12:23:40.516 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:40 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1[34241]: 2024-02-07T12:23:40.188+0000 7ffb8e7d8700 -1 osd.1 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-07T12:23:45.873 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:45 smithi106 podman[43200]: 2024-02-07 12:23:45.606255495 +0000 UTC m=+6.437730628 container died d41ffd28e7b692a517d2ce3568c672ec05371f73491d88aa89a84706c83e6a93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131) 2024-02-07T12:23:46.240 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:45 smithi106 podman[43200]: 2024-02-07 12:23:45.962753451 +0000 UTC m=+6.794228630 container cleanup d41ffd28e7b692a517d2ce3568c672ec05371f73491d88aa89a84706c83e6a93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0) 2024-02-07T12:23:46.240 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:45 smithi106 bash[43200]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1 2024-02-07T12:23:46.516 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:46 smithi106 podman[43223]: 2024-02-07 12:23:46.23885826 +0000 UTC m=+0.628033564 container remove d41ffd28e7b692a517d2ce3568c672ec05371f73491d88aa89a84706c83e6a93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T12:23:46.876 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:46 smithi106 podman[43321]: 2024-02-07 12:23:46.473925559 +0000 UTC m=+0.017607546 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T12:23:46.877 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:46 smithi106 podman[43321]: 2024-02-07 12:23:46.572880311 +0000 UTC m=+0.116562298 container create c97e038552cf5eb0fa9e958ef16bd3ef12f741675db3ea6d8b035a80b0f3f4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-02-07T12:23:46.877 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:46 smithi106 podman[43321]: 2024-02-07 12:23:46.791812435 +0000 UTC m=+0.335494418 container init c97e038552cf5eb0fa9e958ef16bd3ef12f741675db3ea6d8b035a80b0f3f4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T12:23:46.877 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:46 smithi106 podman[43321]: 2024-02-07 12:23:46.797810464 +0000 UTC m=+0.341492457 container start c97e038552cf5eb0fa9e958ef16bd3ef12f741675db3ea6d8b035a80b0f3f4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-07T12:23:46.877 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:46 smithi106 podman[43321]: 2024-02-07 12:23:46.874978633 +0000 UTC m=+0.418660630 container attach c97e038552cf5eb0fa9e958ef16bd3ef12f741675db3ea6d8b035a80b0f3f4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2) 2024-02-07T12:23:47.266 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:47 smithi106 podman[43321]: 2024-02-07 12:23:47.08522959 +0000 UTC m=+0.628911575 container died c97e038552cf5eb0fa9e958ef16bd3ef12f741675db3ea6d8b035a80b0f3f4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, ceph=True) 2024-02-07T12:23:47.698 INFO:journalctl@ceph.osd.1.smithi106.stdout:Feb 07 12:23:47 smithi106 podman[43367]: 2024-02-07 12:23:47.677854588 +0000 UTC m=+0.587168148 container remove c97e038552cf5eb0fa9e958ef16bd3ef12f741675db3ea6d8b035a80b0f3f4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-07T12:23:47.703 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@osd.1.service' 2024-02-07T12:23:47.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:23:47.744 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-07T12:23:47.744 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-07T12:23:47.744 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@osd.2 2024-02-07T12:23:48.162 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:47 smithi171 systemd[1]: Stopping Ceph osd.2 for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:48.662 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:48 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2[35269]: 2024-02-07T12:23:48.373+0000 7f627777c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T12:23:48.662 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:48 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2[35269]: 2024-02-07T12:23:48.373+0000 7f627777c700 -1 osd.2 31 *** Got signal Terminated *** 2024-02-07T12:23:48.662 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:48 smithi171 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2[35269]: 2024-02-07T12:23:48.373+0000 7f627777c700 -1 osd.2 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-07T12:23:54.088 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:53 smithi171 podman[43939]: 2024-02-07 12:23:53.820652146 +0000 UTC m=+5.879283898 container died 0c2fe4194f3367f69377f6e4b088a7eda17e01db1d0050175ac1a879062c08b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T12:23:54.089 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:54 smithi171 podman[43939]: 2024-02-07 12:23:54.086715052 +0000 UTC m=+6.145346803 container cleanup 0c2fe4194f3367f69377f6e4b088a7eda17e01db1d0050175ac1a879062c08b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T12:23:54.411 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:54 smithi171 bash[43939]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2 2024-02-07T12:23:54.412 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:54 smithi171 podman[43961]: 2024-02-07 12:23:54.312584407 +0000 UTC m=+0.491285748 container remove 0c2fe4194f3367f69377f6e4b088a7eda17e01db1d0050175ac1a879062c08b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-07T12:23:54.900 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:54 smithi171 podman[44058]: 2024-02-07 12:23:54.507352102 +0000 UTC m=+0.018355860 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T12:23:54.900 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:54 smithi171 podman[44058]: 2024-02-07 12:23:54.655314897 +0000 UTC m=+0.166318648 container create 2967b995137b78c8f7eca27db363cbe931ee806a593d3a662b40b3caf2ad2fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T12:23:54.900 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:54 smithi171 podman[44058]: 2024-02-07 12:23:54.832120058 +0000 UTC m=+0.343123809 container init 2967b995137b78c8f7eca27db363cbe931ee806a593d3a662b40b3caf2ad2fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, ceph=True) 2024-02-07T12:23:54.900 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:54 smithi171 podman[44058]: 2024-02-07 12:23:54.837446766 +0000 UTC m=+0.348450517 container start 2967b995137b78c8f7eca27db363cbe931ee806a593d3a662b40b3caf2ad2fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-07T12:23:54.900 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:54 smithi171 podman[44058]: 2024-02-07 12:23:54.898501234 +0000 UTC m=+0.409504995 container attach 2967b995137b78c8f7eca27db363cbe931ee806a593d3a662b40b3caf2ad2fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True) 2024-02-07T12:23:55.162 INFO:journalctl@ceph.osd.2.smithi171.stdout:Feb 07 12:23:55 smithi171 podman[44058]: 2024-02-07 12:23:55.125761711 +0000 UTC m=+0.636765473 container died 2967b995137b78c8f7eca27db363cbe931ee806a593d3a662b40b3caf2ad2fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.license=GPLv2) 2024-02-07T12:23:55.653 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@osd.2.service' 2024-02-07T12:23:55.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:23:55.694 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-07T12:23:55.694 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-07T12:23:55.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@rgw.foo.a 2024-02-07T12:23:55.991 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:23:55 smithi078 systemd[1]: Stopping Ceph rgw.foo.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:23:56.381 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:23:55 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a[46317]: 2024-02-07T12:23:55.989+0000 7f2ada0b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T12:23:56.381 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:23:55 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a[46317]: 2024-02-07T12:23:55.989+0000 7f2ae8922ac0 -1 shutting down 2024-02-07T12:24:06.314 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:24:05 smithi078 bash[56362]: time="2024-02-07T12:24:05Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-07T12:24:06.315 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:24:06 smithi078 podman[56362]: 2024-02-07 12:24:06.009897945 +0000 UTC m=+10.127036562 container died 1c13524bc9740293b7cdb53a3577d0dc16e8955cd01b09eeae0a044dacdc410d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T12:24:06.615 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:24:06 smithi078 podman[56362]: 2024-02-07 12:24:06.499969879 +0000 UTC m=+10.617108540 container cleanup 1c13524bc9740293b7cdb53a3577d0dc16e8955cd01b09eeae0a044dacdc410d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD) 2024-02-07T12:24:06.615 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:24:06 smithi078 bash[56362]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a 2024-02-07T12:24:06.859 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@rgw.foo.a.service' 2024-02-07T12:24:06.881 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:24:06 smithi078 podman[56399]: 2024-02-07 12:24:06.754288833 +0000 UTC m=+0.736294111 container remove 1c13524bc9740293b7cdb53a3577d0dc16e8955cd01b09eeae0a044dacdc410d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, CEPH_POINT_RELEASE=) 2024-02-07T12:24:06.881 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:24:06 smithi078 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-02-07T12:24:06.881 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:24:06 smithi078 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2024-02-07T12:24:06.881 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:24:06 smithi078 systemd[1]: Stopped Ceph rgw.foo.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:24:06.881 INFO:journalctl@ceph.rgw.foo.a.smithi078.stdout:Feb 07 12:24:06 smithi078 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@rgw.foo.a.service: Consumed 1.380s CPU time. 2024-02-07T12:24:06.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:24:06.897 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-07T12:24:06.897 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-07T12:24:06.897 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@prometheus.a 2024-02-07T12:24:07.191 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:06 smithi106 systemd[1]: Stopping Ceph prometheus.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.189Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-07T12:24:07.474 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-07T12:24:07.475 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a[40602]: ts=2024-02-07T12:24:07.190Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-07T12:24:07.475 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 podman[43451]: 2024-02-07 12:24:07.200156729 +0000 UTC m=+0.112798283 container died 74e62edb26dca7cb20c3617b5bf3f3728255c8a8db360a4db0d0e6196a9ed452 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T12:24:07.766 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 podman[43451]: 2024-02-07 12:24:07.570424034 +0000 UTC m=+0.483065597 container cleanup 74e62edb26dca7cb20c3617b5bf3f3728255c8a8db360a4db0d0e6196a9ed452 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T12:24:07.766 INFO:journalctl@ceph.prometheus.a.smithi106.stdout:Feb 07 12:24:07 smithi106 bash[43451]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-prometheus-a 2024-02-07T12:24:07.927 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@prometheus.a.service' 2024-02-07T12:24:07.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T12:24:07.964 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-07T12:24:07.964 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-07T12:24:08.161 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:24:09.131 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:24:09 smithi078 systemd[1]: Stopping Ceph alertmanager.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:24:09.527 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:24:09 smithi078 ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a[53681]: ts=2024-02-07T12:24:09.252Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-07T12:24:09.527 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:24:09 smithi078 podman[56720]: 2024-02-07 12:24:09.273120565 +0000 UTC m=+0.119583287 container died 135d6b122780e3f66c4058a73f0927b036523263884a667eab03e67c7f57a899 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T12:24:09.802 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:24:09 smithi078 podman[56720]: 2024-02-07 12:24:09.615994535 +0000 UTC m=+0.462457260 container cleanup 135d6b122780e3f66c4058a73f0927b036523263884a667eab03e67c7f57a899 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T12:24:09.802 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:24:09 smithi078 bash[56720]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a 2024-02-07T12:24:10.061 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:24:09 smithi078 podman[56739]: 2024-02-07 12:24:09.800430175 +0000 UTC m=+0.522479918 container remove 135d6b122780e3f66c4058a73f0927b036523263884a667eab03e67c7f57a899 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T12:24:10.062 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:24:09 smithi078 podman[56739]: 2024-02-07 12:24:09.87596182 +0000 UTC m=+0.598011497 volume remove 2ae9dd0de1ff842a3cddce02f3fbf4bb5b2704db6b2ac97988aba7043311bf16 2024-02-07T12:24:10.381 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:24:10 smithi078 systemd[1]: Stopping Ceph node-exporter.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:24:10.381 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:24:10 smithi078 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-07T12:24:10.381 INFO:journalctl@ceph.alertmanager.a.smithi078.stdout:Feb 07 12:24:10 smithi078 systemd[1]: Stopped Ceph alertmanager.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:24:10.806 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:24:10 smithi078 podman[56855]: 2024-02-07 12:24:10.550259546 +0000 UTC m=+0.129450405 container died fd764afa2f688969a248925b70438ad17e5e17c7af3bd80ce4376089acc56e3a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T12:24:11.130 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:24:10 smithi078 podman[56855]: 2024-02-07 12:24:10.995527668 +0000 UTC m=+0.574718561 container cleanup fd764afa2f688969a248925b70438ad17e5e17c7af3bd80ce4376089acc56e3a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T12:24:11.131 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:24:10 smithi078 bash[56855]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a 2024-02-07T12:24:11.601 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:24:11 smithi078 podman[56874]: 2024-02-07 12:24:11.321868218 +0000 UTC m=+0.776859485 container remove fd764afa2f688969a248925b70438ad17e5e17c7af3bd80ce4376089acc56e3a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T12:24:11.601 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:24:11 smithi078 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-07T12:24:11.601 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:24:11 smithi078 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-07T12:24:11.601 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:24:11 smithi078 systemd[1]: Stopped Ceph node-exporter.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:24:11.601 INFO:journalctl@ceph.node-exporter.a.smithi078.stdout:Feb 07 12:24:11 smithi078 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.a.service: Consumed 1.754s CPU time. 2024-02-07T12:24:12.096 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-07T12:24:12.286 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:24:13.266 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:24:13 smithi106 systemd[1]: Stopping Ceph grafana.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:24:13.692 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:24:13 smithi106 podman[43803]: 2024-02-07 12:24:13.387624837 +0000 UTC m=+0.119769619 container died 517a14cc177120cc0bab575029e930b7ad7b73479a270e2c07e1054fba7be6c4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, vendor=Red Hat, Inc., maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, io.openshift.tags=base rhel8, vcs-type=git, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-02-07T12:24:13.693 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:24:13 smithi106 podman[43803]: 2024-02-07 12:24:13.690431287 +0000 UTC m=+0.422576074 container cleanup 517a14cc177120cc0bab575029e930b7ad7b73479a270e2c07e1054fba7be6c4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a, distribution-scope=public, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , vcs-type=git, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.expose-services=, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-02-07T12:24:14.015 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:24:13 smithi106 bash[43803]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a 2024-02-07T12:24:14.016 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:24:13 smithi106 podman[43822]: 2024-02-07 12:24:13.891620272 +0000 UTC m=+0.506691190 container remove 517a14cc177120cc0bab575029e930b7ad7b73479a270e2c07e1054fba7be6c4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-grafana-a, vendor=Red Hat, Inc., description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, release=239.1651231664, version=8.5, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , io.buildah.version=1.32.0, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, build-date=2022-04-29T11:48:11.055897, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, architecture=x86_64) 2024-02-07T12:24:14.016 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:24:13 smithi106 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-07T12:24:14.475 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:24:14 smithi106 systemd[1]: Stopping Ceph node-exporter.b for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:24:14.475 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:24:14 smithi106 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-07T12:24:14.476 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:24:14 smithi106 systemd[1]: Stopped Ceph grafana.a for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:24:14.476 INFO:journalctl@ceph.grafana.a.smithi106.stdout:Feb 07 12:24:14 smithi106 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@grafana.a.service: Consumed 2.738s CPU time. 2024-02-07T12:24:14.766 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:24:14 smithi106 podman[43941]: 2024-02-07 12:24:14.473893001 +0000 UTC m=+0.115669470 container died bbf751681d39d7f9c5ff2bb83add51fde8e8aab2bdfd63d45ce38103443770ba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T12:24:15.198 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:24:14 smithi106 podman[43941]: 2024-02-07 12:24:14.802698809 +0000 UTC m=+0.444475280 container cleanup bbf751681d39d7f9c5ff2bb83add51fde8e8aab2bdfd63d45ce38103443770ba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T12:24:15.198 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:24:14 smithi106 bash[43941]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b 2024-02-07T12:24:15.198 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:24:14 smithi106 podman[43960]: 2024-02-07 12:24:14.986808033 +0000 UTC m=+0.516160727 container remove bbf751681d39d7f9c5ff2bb83add51fde8e8aab2bdfd63d45ce38103443770ba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T12:24:15.198 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:24:14 smithi106 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-07T12:24:15.515 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:24:15 smithi106 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-07T12:24:15.516 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:24:15 smithi106 systemd[1]: Stopped Ceph node-exporter.b for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:24:15.516 INFO:journalctl@ceph.node-exporter.b.smithi106.stdout:Feb 07 12:24:15 smithi106 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.b.service: Consumed 1.690s CPU time. 2024-02-07T12:24:15.857 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-07T12:24:16.048 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:24:16.912 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:24:16 smithi171 systemd[1]: Stopping Ceph node-exporter.c for dc35b7c8-c5b2-11ee-95b6-87774f69a715... 2024-02-07T12:24:17.271 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:24:16 smithi171 podman[44362]: 2024-02-07 12:24:16.990209166 +0000 UTC m=+0.122853276 container died 54cc3d85a17b7d65fb0deb5bb022589a6745b2ce1d7cbf45f7b49027efd6191b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T12:24:17.611 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:24:17 smithi171 podman[44362]: 2024-02-07 12:24:17.342668712 +0000 UTC m=+0.475312821 container cleanup 54cc3d85a17b7d65fb0deb5bb022589a6745b2ce1d7cbf45f7b49027efd6191b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T12:24:17.612 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:24:17 smithi171 bash[44362]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c 2024-02-07T12:24:17.912 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:24:17 smithi171 podman[44383]: 2024-02-07 12:24:17.610412137 +0000 UTC m=+0.625087514 container remove 54cc3d85a17b7d65fb0deb5bb022589a6745b2ce1d7cbf45f7b49027efd6191b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T12:24:17.912 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:24:17 smithi171 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.c.service: Main process exited, code=exited, status=143/n/a 2024-02-07T12:24:17.912 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:24:17 smithi171 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.c.service: Failed with result 'exit-code'. 2024-02-07T12:24:17.912 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:24:17 smithi171 systemd[1]: Stopped Ceph node-exporter.c for dc35b7c8-c5b2-11ee-95b6-87774f69a715. 2024-02-07T12:24:17.912 INFO:journalctl@ceph.node-exporter.c.smithi171.stdout:Feb 07 12:24:17 smithi171 systemd[1]: ceph-dc35b7c8-c5b2-11ee-95b6-87774f69a715@node-exporter.c.service: Consumed 1.778s CPU time. 2024-02-07T12:24:18.417 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T12:24:18.451 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T12:24:18.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T12:24:18.515 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-07T12:24:18.517 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/remote/smithi078/crash 2024-02-07T12:24:18.517 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/crash -- . 2024-02-07T12:24:18.569 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-07T12:24:18.569 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-02-07T12:24:18.571 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/remote/smithi106/crash 2024-02-07T12:24:18.572 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/crash -- . 2024-02-07T12:24:18.614 INFO:teuthology.orchestra.run.smithi106.stderr:tar: /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-07T12:24:18.614 INFO:teuthology.orchestra.run.smithi106.stderr:tar: Error is not recoverable: exiting now 2024-02-07T12:24:18.616 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/remote/smithi171/crash 2024-02-07T12:24:18.617 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/crash -- . 2024-02-07T12:24:18.661 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-07T12:24:18.661 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-02-07T12:24:18.662 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-07T12:24:18.662 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T12:24:18.693 INFO:teuthology.orchestra.run.smithi078.stdout:2024-02-07T12:20:24.995977+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T12:24:18.693 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-07T12:24:18.693 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[SEC\]' /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T12:24:18.763 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]' /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T12:24:18.833 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[WRN\]' /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T12:24:18.902 INFO:teuthology.orchestra.run.smithi078.stdout:2024-02-07T12:20:24.995977+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T12:24:18.902 INFO:tasks.cephadm:Compressing logs... 2024-02-07T12:24:18.902 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-07T12:24:18.945 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-07T12:24:18.947 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-07T12:24:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-02-07T12:24:18.970 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-02-07T12:24:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mon.a.log 2024-02-07T12:24:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-07T12:24:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-07T12:24:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mgr.y.log 2024-02-07T12:24:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log: --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-osd.0.log 2024-02-07T12:24:18.972 INFO:teuthology.orchestra.run.smithi078.stderr: 57.6% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-07T12:24:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-07T12:24:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log 2024-02-07T12:24:18.972 INFO:teuthology.orchestra.run.smithi078.stderr: 89.3% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-07T12:24:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-07T12:24:18.972 INFO:teuthology.orchestra.run.smithi078.stderr: 90.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-07T12:24:18.972 INFO:teuthology.orchestra.run.smithi078.stderr: 82.5% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-07T12:24:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log: /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-volume.log: 84.9% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-07T12:24:18.974 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-07T12:24:18.974 INFO:teuthology.orchestra.run.smithi171.stderr:: No such file or directory 2024-02-07T12:24:18.974 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-07T12:24:18.974 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mgr.z.log 2024-02-07T12:24:18.974 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-07T12:24:18.976 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log 2024-02-07T12:24:18.976 INFO:teuthology.orchestra.run.smithi171.stderr: 90.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-07T12:24:18.976 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-osd.2.log 2024-02-07T12:24:18.976 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.cephadm.log: /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log: 81.0% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-07T12:24:18.976 INFO:teuthology.orchestra.run.smithi171.stderr: 89.4% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-07T12:24:18.977 INFO:teuthology.orchestra.run.smithi078.stderr: 90.5% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-07T12:24:18.978 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-07T12:24:18.979 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mon.c.log 2024-02-07T12:24:18.979 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-osd.2.log: 84.6% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-07T12:24:18.979 INFO:teuthology.orchestra.run.smithi106.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-07T12:24:18.979 INFO:teuthology.orchestra.run.smithi106.stderr:: No such file or directory 2024-02-07T12:24:18.979 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-07T12:24:18.979 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-osd.1.log 2024-02-07T12:24:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mgr.x.log 2024-02-07T12:24:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.audit.log: /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-07T12:24:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log 2024-02-07T12:24:18.981 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-volume.log: /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mon.c.log: 90.5% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-07T12:24:18.981 INFO:teuthology.orchestra.run.smithi171.stderr: 90.2% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mgr.z.log.gz 2024-02-07T12:24:18.982 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-07T12:24:18.982 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-07T12:24:18.982 INFO:teuthology.orchestra.run.smithi106.stderr: 81.0% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-07T12:24:18.982 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mon.b.log 2024-02-07T12:24:18.982 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-volume.log: 81.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-07T12:24:18.982 INFO:teuthology.orchestra.run.smithi106.stderr: 84.4% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-07T12:24:18.982 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mon.b.log: 90.5% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-07T12:24:18.982 INFO:teuthology.orchestra.run.smithi106.stderr: 90.2% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mgr.x.log.gz 2024-02-07T12:24:19.031 INFO:teuthology.orchestra.run.smithi171.stderr: 92.3% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mon.c.log.gz 2024-02-07T12:24:19.032 INFO:teuthology.orchestra.run.smithi106.stderr: 92.3% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mon.b.log.gz 2024-02-07T12:24:19.047 INFO:teuthology.orchestra.run.smithi078.stderr: 88.9% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mgr.y.log.gz 2024-02-07T12:24:19.147 INFO:teuthology.orchestra.run.smithi078.stderr: 91.4% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-mon.a.log.gz 2024-02-07T12:24:19.232 INFO:teuthology.orchestra.run.smithi078.stderr: 94.3% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-osd.0.log.gz 2024-02-07T12:24:19.234 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-02-07T12:24:19.234 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m0.277s 2024-02-07T12:24:19.234 INFO:teuthology.orchestra.run.smithi078.stderr:user 0m0.509s 2024-02-07T12:24:19.234 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m0.046s 2024-02-07T12:24:19.235 INFO:teuthology.orchestra.run.smithi171.stderr: 94.3% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-osd.2.log.gz 2024-02-07T12:24:19.237 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-02-07T12:24:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m0.276s 2024-02-07T12:24:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:user 0m0.327s 2024-02-07T12:24:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m0.031s 2024-02-07T12:24:19.272 INFO:teuthology.orchestra.run.smithi106.stderr: 94.2% -- replaced with /var/log/ceph/dc35b7c8-c5b2-11ee-95b6-87774f69a715/ceph-osd.1.log.gz 2024-02-07T12:24:19.274 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-02-07T12:24:19.275 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m0.312s 2024-02-07T12:24:19.275 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m0.363s 2024-02-07T12:24:19.275 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m0.034s 2024-02-07T12:24:19.275 INFO:tasks.cephadm:Archiving logs... 2024-02-07T12:24:19.275 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/remote/smithi078/log 2024-02-07T12:24:19.276 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-07T12:24:19.404 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/remote/smithi106/log 2024-02-07T12:24:19.405 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-07T12:24:19.494 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/remote/smithi171/log 2024-02-07T12:24:19.495 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-07T12:24:19.580 INFO:tasks.cephadm:Removing cluster... 2024-02-07T12:24:19.580 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 --force 2024-02-07T12:24:19.757 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:24:19.871 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 --force 2024-02-07T12:24:20.053 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:24:20.167 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc35b7c8-c5b2-11ee-95b6-87774f69a715 --force 2024-02-07T12:24:20.358 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: dc35b7c8-c5b2-11ee-95b6-87774f69a715 2024-02-07T12:24:20.473 INFO:tasks.cephadm:Removing cephadm ... 2024-02-07T12:24:20.474 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-07T12:24:20.489 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-07T12:24:20.504 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-07T12:24:20.519 INFO:tasks.cephadm:Teardown complete 2024-02-07T12:24:20.519 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-02-07T12:24:20.531 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi078:/dev/vg_nvme/lv_1... 2024-02-07T12:24:20.531 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme disconnect -n lv_1 2024-02-07T12:24:20.677 INFO:teuthology.orchestra.run.smithi078.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-07T12:24:20.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:20.679 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi078:/dev/vg_nvme/lv_2... 2024-02-07T12:24:20.679 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme disconnect -n lv_2 2024-02-07T12:24:20.820 INFO:teuthology.orchestra.run.smithi078.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-07T12:24:20.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:20.822 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi078:/dev/vg_nvme/lv_3... 2024-02-07T12:24:20.822 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme disconnect -n lv_3 2024-02-07T12:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-07T12:24:20.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:20.979 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi078:/dev/vg_nvme/lv_4... 2024-02-07T12:24:20.979 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme disconnect -n lv_4 2024-02-07T12:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-07T12:24:21.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:21.139 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-07T12:24:21.139 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/scratch_devs 2024-02-07T12:24:21.166 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi106:/dev/vg_nvme/lv_1... 2024-02-07T12:24:21.166 DEBUG:teuthology.orchestra.run.smithi106:> sudo nvme disconnect -n lv_1 2024-02-07T12:24:21.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:21.307 INFO:teuthology.orchestra.run.smithi106.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-07T12:24:21.308 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi106:/dev/vg_nvme/lv_2... 2024-02-07T12:24:21.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo nvme disconnect -n lv_2 2024-02-07T12:24:21.455 INFO:teuthology.orchestra.run.smithi106.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-07T12:24:21.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:21.456 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi106:/dev/vg_nvme/lv_3... 2024-02-07T12:24:21.457 DEBUG:teuthology.orchestra.run.smithi106:> sudo nvme disconnect -n lv_3 2024-02-07T12:24:21.606 INFO:teuthology.orchestra.run.smithi106.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-07T12:24:21.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:21.608 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi106:/dev/vg_nvme/lv_4... 2024-02-07T12:24:21.608 DEBUG:teuthology.orchestra.run.smithi106:> sudo nvme disconnect -n lv_4 2024-02-07T12:24:21.795 INFO:teuthology.orchestra.run.smithi106.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-07T12:24:21.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:21.797 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-02-07T12:24:21.797 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/scratch_devs 2024-02-07T12:24:21.824 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi171:/dev/vg_nvme/lv_1... 2024-02-07T12:24:21.824 DEBUG:teuthology.orchestra.run.smithi171:> sudo nvme disconnect -n lv_1 2024-02-07T12:24:21.963 INFO:teuthology.orchestra.run.smithi171.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-07T12:24:21.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:21.966 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi171:/dev/vg_nvme/lv_2... 2024-02-07T12:24:21.966 DEBUG:teuthology.orchestra.run.smithi171:> sudo nvme disconnect -n lv_2 2024-02-07T12:24:22.116 INFO:teuthology.orchestra.run.smithi171.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-07T12:24:22.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:22.118 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi171:/dev/vg_nvme/lv_3... 2024-02-07T12:24:22.118 DEBUG:teuthology.orchestra.run.smithi171:> sudo nvme disconnect -n lv_3 2024-02-07T12:24:22.287 INFO:teuthology.orchestra.run.smithi171.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-07T12:24:22.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:22.289 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi171:/dev/vg_nvme/lv_4... 2024-02-07T12:24:22.289 DEBUG:teuthology.orchestra.run.smithi171:> sudo nvme disconnect -n lv_4 2024-02-07T12:24:22.445 INFO:teuthology.orchestra.run.smithi171.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-07T12:24:22.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:22.447 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-02-07T12:24:22.447 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/scratch_devs 2024-02-07T12:24:22.476 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-07T12:24:22.489 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-07T12:24:22.490 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T12:24:22.492 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T12:24:22.495 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T12:24:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-02-07T12:24:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-02-07T12:24:22.509 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T12:24:22.509 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-02-07T12:24:22.509 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 -601us[ -593us] +/- 97ms 2024-02-07T12:24:22.509 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -524us[ -516us] +/- 68ms 2024-02-07T12:24:22.509 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +1367us[+1367us] +/- 92ms 2024-02-07T12:24:22.509 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:24:22.511 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T12:24:22.511 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-02-07T12:24:22.511 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 -612us[ -580us] +/- 97ms 2024-02-07T12:24:22.511 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -519us[ -488us] +/- 68ms 2024-02-07T12:24:22.511 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +1347us[+1347us] +/- 92ms 2024-02-07T12:24:22.511 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:24:22.532 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-02-07T12:24:22.536 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T12:24:22.536 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-02-07T12:24:22.536 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 -611us[ -611us] +/- 97ms 2024-02-07T12:24:22.536 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -528us[ -504us] +/- 68ms 2024-02-07T12:24:22.536 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +1349us[+1373us] +/- 92ms 2024-02-07T12:24:22.536 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T12:24:22.537 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-07T12:24:22.546 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-07T12:24:22.548 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-07T12:24:22.557 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-07T12:24:22.630 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-07T12:24:22.697 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-07T12:24:22.770 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T12:24:22.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:22.799 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T12:24:22.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:22.829 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T12:24:22.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:22.861 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-07T12:24:22.870 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-07T12:24:22.879 INFO:teuthology.task.internal:Duration was 681.535556 seconds 2024-02-07T12:24:22.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-07T12:24:22.888 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-07T12:24:22.889 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-07T12:24:22.891 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-07T12:24:22.893 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-07T12:24:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T12:24:22.933 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T12:24:22.947 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T12:24:23.427 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-07T12:24:23.427 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-02-07T12:24:23.427 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-07T12:24:23.486 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-02-07T12:24:23.487 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-07T12:24:23.514 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-02-07T12:24:23.515 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-07T12:24:23.546 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-07T12:24:23.546 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-07T12:24:23.548 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-07T12:24:23.557 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-07T12:24:23.616 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-07T12:24:23.616 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-07T12:24:23.618 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-07T12:24:23.626 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-07T12:24:23.804 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-07T12:24:23.814 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-07T12:24:23.814 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-07T12:24:23.840 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-07T12:24:23.867 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-07T12:24:23.895 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-07T12:24:23.906 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-07T12:24:23.908 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-07T12:24:23.910 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-07T12:24:23.931 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-02-07T12:24:23.933 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-02-07T12:24:23.960 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-02-07T12:24:23.980 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-07T12:24:24.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:24.008 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-07T12:24:24.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:24.022 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-07T12:24:24.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:24:24.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-07T12:24:24.046 INFO:teuthology.task.internal:Transferring archived files... 2024-02-07T12:24:24.047 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/remote/smithi078 2024-02-07T12:24:24.047 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-07T12:24:24.086 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/remote/smithi106 2024-02-07T12:24:24.086 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-07T12:24:24.123 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731/remote/smithi171 2024-02-07T12:24:24.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-07T12:24:24.158 INFO:teuthology.task.internal:Removing archive directory... 2024-02-07T12:24:24.158 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-07T12:24:24.161 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-07T12:24:24.163 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-07T12:24:24.208 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-07T12:24:24.221 INFO:teuthology.task.internal:Not uploading archives. 2024-02-07T12:24:24.221 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-07T12:24:24.237 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-07T12:24:24.238 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-07T12:24:24.240 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-07T12:24:24.242 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-07T12:24:24.252 INFO:teuthology.orchestra.run.smithi078.stdout: 266030 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 7 12:24 /home/ubuntu/cephtest 2024-02-07T12:24:24.254 INFO:teuthology.orchestra.run.smithi106.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 7 12:24 /home/ubuntu/cephtest 2024-02-07T12:24:24.263 INFO:teuthology.orchestra.run.smithi171.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 7 12:24 /home/ubuntu/cephtest 2024-02-07T12:24:24.264 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-07T12:24:24.275 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-07T12:24:24.298 INFO:teuthology.nuke:Checking targets against current locks 2024-02-07T12:24:24.330 DEBUG:teuthology.nuke:shortname: smithi078 2024-02-07T12:24:24.331 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T12:24:24.357 DEBUG:teuthology.nuke:shortname: smithi106 2024-02-07T12:24:24.357 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T12:24:24.362 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:03:51.514475', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGGzjAw/GvhXBhlnFu+gD8gFQGNZF//ng8zqgP/wRhJX9aRgUCboL3ELhBsXvRvRVT6mYSztKWOAr7qM6lV1WI='} 2024-02-07T12:24:24.379 DEBUG:teuthology.nuke:shortname: smithi171 2024-02-07T12:24:24.380 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T12:24:24.385 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:03:51.515692', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgg54BK6U3EbSSrDWNc7km5nWVlgFlbxYqu41t1/gKZUKN1a+B4LvjqNy0fd24sodoJ4JVAbQadKyA6UJjEO7M='} 2024-02-07T12:24:24.400 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548731', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:03:51.516664', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNtRBEv0PJalB1W6PuKp2BrtE85xVRvqBzFrRTI93cIBXDj7UweVgYon6xubQqvB7sVtICOhWgGsQITHs2O0Nnk='} 2024-02-07T12:24:24.417 INFO:teuthology.orchestra.console.smithi078:Power off 2024-02-07T12:24:24.418 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-07T12:24:24.442 DEBUG:teuthology.orchestra.console.smithi078:power off output: Chassis Power Control: Down/Off 2024-02-07T12:24:24.442 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:24.459 INFO:teuthology.orchestra.console.smithi171:Power off 2024-02-07T12:24:24.460 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-07T12:24:24.472 INFO:teuthology.orchestra.console.smithi106:Power off 2024-02-07T12:24:24.472 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-07T12:24:24.482 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2024-02-07T12:24:24.482 DEBUG:teuthology.orchestra.console.smithi171:power off output: Chassis Power Control: Down/Off 2024-02-07T12:24:24.482 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:24.496 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2024-02-07T12:24:24.496 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:24.504 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-02-07T12:24:24.519 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-02-07T12:24:28.482 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:28.506 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:28.520 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:28.597 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2024-02-07T12:24:28.616 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-02-07T12:24:28.630 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-02-07T12:24:32.600 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:32.617 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:32.630 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:32.711 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2024-02-07T12:24:32.727 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is off 2024-02-07T12:24:32.740 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-02-07T12:24:32.828 INFO:teuthology.orchestra.console.smithi171:Power off completed 2024-02-07T12:24:32.841 INFO:teuthology.orchestra.console.smithi106:Power off completed 2024-02-07T12:24:36.716 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T12:24:36.827 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2024-02-07T12:24:36.927 INFO:teuthology.orchestra.console.smithi078:Power off completed 2024-02-07T12:24:37.031 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} duration: 681.5355560779572 failure_reason: '"2024-02-07T12:20:24.995977+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' owner: scheduled_gabrioux@teuthology success: false 2024-02-07T12:24:37.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-07T12:24:37.122 INFO:teuthology.run:FAIL