2024-02-14T17:04:23.707 INFO:root:teuthology version: 0.0.1.dev258+g9e44c16 2024-02-14T17:04:23.707 DEBUG:teuthology.run:Teuthology command: teuthology --name gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi --owner scheduled_gabrioux@teuthology --verbose --archive /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760 --description orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} -- /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/orig.config.yaml 2024-02-14T17:04:23.735 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T17:04:23.811 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760 branch: wip-guits-testing-2024-02-14-1033 description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} email: gabrioux@ibm.com first_in_suite: false job_id: '7559760' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-2024-02-14-1033 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-2024-02-14-1033 sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 4718 sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-2024-02-14-1033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8727ce939fc044a98f215e29696bb1433f5758d1/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 targets: smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKh/VHUWQOqI6QPvCO7+14T/bSfcmUqXvrmmQp+6+fr7zVu8bTlQ+0o/m13/Fi4TFxD51bUN7n56Dvxj+2u4ht0= smithi091.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBk4YGW62+4YkJfAbNOEeGrLY1Z5VeUfQHlDS5XTU1sJLPwgxW/+R3eGh9gGUKgQckk+VIjqrwZC5w3KPfz65Lk= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAl3UarpUpAr+yoD5/2J0GNylfCVpPOCn3D/R8yGbU/+ukyW9QLXTurAWJnZx1CbygADGYd/mVDvCEvxTurqu20= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9e44c16382dfc4c612f7437cbb344c76223137cc timestamp: 2024-02-14_13:54:10 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-14T17:04:23.811 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8727ce939fc044a98f215e29696bb1433f5758d1/qa; will attempt to use it 2024-02-14T17:04:23.812 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8727ce939fc044a98f215e29696bb1433f5758d1/qa/tasks 2024-02-14T17:04:23.812 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-14T17:04:23.814 INFO:teuthology.task.internal:Checking packages... 2024-02-14T17:04:23.839 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8727ce939fc044a98f215e29696bb1433f5758d1' 2024-02-14T17:04:23.839 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-14T17:04:23.839 INFO:teuthology.packaging:ref: None 2024-02-14T17:04:23.839 INFO:teuthology.packaging:tag: None 2024-02-14T17:04:23.839 INFO:teuthology.packaging:branch: wip-guits-testing-2024-02-14-1033 2024-02-14T17:04:23.839 INFO:teuthology.packaging:sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:04:23.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-2024-02-14-1033 2024-02-14T17:04:23.976 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1430.g8727ce93 2024-02-14T17:04:23.977 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-14T17:04:23.984 INFO:teuthology.task.internal:no buildpackages task found 2024-02-14T17:04:23.984 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-14T17:04:24.015 INFO:teuthology.task.internal:Saving configuration 2024-02-14T17:04:24.027 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-14T17:04:24.033 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T17:04:24.057 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 16:59:15.164873', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKh/VHUWQOqI6QPvCO7+14T/bSfcmUqXvrmmQp+6+fr7zVu8bTlQ+0o/m13/Fi4TFxD51bUN7n56Dvxj+2u4ht0='} 2024-02-14T17:04:24.082 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi091.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 16:59:15.166664', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBk4YGW62+4YkJfAbNOEeGrLY1Z5VeUfQHlDS5XTU1sJLPwgxW/+R3eGh9gGUKgQckk+VIjqrwZC5w3KPfz65Lk='} 2024-02-14T17:04:24.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 16:59:15.162886', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAl3UarpUpAr+yoD5/2J0GNylfCVpPOCn3D/R8yGbU/+ukyW9QLXTurAWJnZx1CbygADGYd/mVDvCEvxTurqu20='} 2024-02-14T17:04:24.105 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-14T17:04:24.111 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-14T17:04:24.111 INFO:teuthology.task.internal:roles: ubuntu@smithi091.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-02-14T17:04:24.111 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-02-14T17:04:24.111 INFO:teuthology.run_tasks:Running task console_log... 2024-02-14T17:04:24.240 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2d5711cf70>, signals=[15]) 2024-02-14T17:04:24.240 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-14T17:04:24.246 INFO:teuthology.task.internal:Opening connections... 2024-02-14T17:04:24.247 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-02-14T17:04:24.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:04:24.338 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi091.front.sepia.ceph.com 2024-02-14T17:04:24.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:04:24.432 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-02-14T17:04:24.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:04:24.528 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-14T17:04:24.534 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-02-14T17:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-02-14T17:04:24.553 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-02-14T17:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-02-14T17:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-02-14T17:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T17:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T17:04:24.612 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-02-14T17:04:24.641 DEBUG:teuthology.orchestra.run.smithi091:> uname -m 2024-02-14T17:04:24.659 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64 2024-02-14T17:04:24.660 DEBUG:teuthology.orchestra.run.smithi091:> cat /etc/os-release 2024-02-14T17:04:24.720 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="CentOS Stream" 2024-02-14T17:04:24.720 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="9" 2024-02-14T17:04:24.720 INFO:teuthology.orchestra.run.smithi091.stdout:ID="centos" 2024-02-14T17:04:24.720 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="rhel fedora" 2024-02-14T17:04:24.720 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="9" 2024-02-14T17:04:24.720 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el9" 2024-02-14T17:04:24.720 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T17:04:24.720 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31" 2024-02-14T17:04:24.721 INFO:teuthology.orchestra.run.smithi091.stdout:LOGO="fedora-logo-icon" 2024-02-14T17:04:24.721 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T17:04:24.721 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://centos.org/" 2024-02-14T17:04:24.721 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T17:04:24.721 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T17:04:24.721 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T17:04:24.721 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2024-02-14T17:04:24.748 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-02-14T17:04:24.765 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-02-14T17:04:24.765 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-02-14T17:04:24.819 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-02-14T17:04:24.819 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-02-14T17:04:24.819 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-02-14T17:04:24.820 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-02-14T17:04:24.820 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-02-14T17:04:24.820 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-02-14T17:04:24.820 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T17:04:24.820 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-02-14T17:04:24.820 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-02-14T17:04:24.820 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T17:04:24.820 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-02-14T17:04:24.820 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T17:04:24.821 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T17:04:24.821 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T17:04:24.821 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-02-14T17:04:24.848 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-14T17:04:24.858 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-14T17:04:24.923 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-14T17:04:24.924 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-02-14T17:04:24.926 DEBUG:teuthology.orchestra.run.smithi091:> test '!' -e /home/ubuntu/cephtest 2024-02-14T17:04:24.929 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-02-14T17:04:24.945 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-14T17:04:24.951 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-14T17:04:24.951 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-02-14T17:04:24.983 DEBUG:teuthology.orchestra.run.smithi091:> test -z $(ls -A /var/lib/ceph) 2024-02-14T17:04:24.989 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-02-14T17:04:25.311 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-14T17:04:25.392 INFO:teuthology.run_tasks:Running task kernel... 2024-02-14T17:04:25.409 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-14T17:04:25.409 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-14T17:04:25.409 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T17:04:25.409 DEBUG:teuthology.orchestra.run.smithi091:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T17:04:25.410 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T17:04:25.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:04:25.426 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-02-14T17:04:25.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:04:25.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:04:25.428 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-02-14T17:04:25.429 DEBUG:teuthology.orchestra.run.smithi091:> uname -r 2024-02-14T17:04:25.482 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-361.el9.x86_64 2024-02-14T17:04:25.483 INFO:teuthology.task.kernel:Running kernel on smithi120: 5.14.0-361.el9.x86_64 2024-02-14T17:04:25.483 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum install -y kernel 2024-02-14T17:04:25.484 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-361.el9.x86_64 2024-02-14T17:04:25.484 INFO:teuthology.task.kernel:Running kernel on smithi082: 5.14.0-361.el9.x86_64 2024-02-14T17:04:25.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y kernel 2024-02-14T17:04:25.489 INFO:teuthology.orchestra.run.smithi091.stdout:5.14.0-361.el9.x86_64 2024-02-14T17:04:25.489 INFO:teuthology.task.kernel:Running kernel on smithi091: 5.14.0-361.el9.x86_64 2024-02-14T17:04:25.490 DEBUG:teuthology.orchestra.run.smithi091:> sudo yum install -y kernel 2024-02-14T17:04:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - AppStream 8.9 MB/s | 6.7 MB 00:00 2024-02-14T17:04:27.861 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - AppStream 9.7 MB/s | 6.7 MB 00:00 2024-02-14T17:04:28.445 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS-9 - AppStream 4.7 MB/s | 6.7 MB 00:01 2024-02-14T17:04:29.503 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-02-14T17:04:29.528 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-02-14T17:04:29.845 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-02-14T17:04:31.836 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-02-14T17:04:31.918 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-02-14T17:04:33.260 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.0 MB 00:03 2024-02-14T17:04:34.422 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-02-14T17:04:34.969 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-02-14T17:04:37.535 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 19 MB 00:04 2024-02-14T17:04:39.170 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.1 MB 00:00 2024-02-14T17:04:40.778 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-02-14T17:04:40.812 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.1 MB 00:02 2024-02-14T17:04:41.427 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-02-14T17:04:43.129 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 15 kB 00:00 2024-02-14T17:04:43.171 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 15 kB 00:01 2024-02-14T17:04:44.471 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2024-02-14T17:04:47.030 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 20 MB 00:03 2024-02-14T17:04:49.345 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-02-14T17:04:51.627 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-14T17:04:51.627 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-14T17:04:51.721 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout: kernel x86_64 5.14.0-420.el9 CentOS-BaseOS 6.0 M 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-core x86_64 5.14.0-420.el9 CentOS-BaseOS 21 M 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules x86_64 5.14.0-420.el9 CentOS-BaseOS 39 M 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules-core x86_64 5.14.0-420.el9 CentOS-BaseOS 33 M 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-02-14T17:04:51.729 INFO:teuthology.orchestra.run.smithi082.stdout:Install 4 Packages 2024-02-14T17:04:51.730 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:04:51.730 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 99 M 2024-02-14T17:04:51.730 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 122 M 2024-02-14T17:04:51.730 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-02-14T17:04:51.897 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-02-14T17:04:52.138 INFO:teuthology.orchestra.run.smithi082.stdout:(1/4): kernel-5.14.0-420.el9.x86_64.rpm 15 MB/s | 6.0 MB 00:00 2024-02-14T17:04:52.847 INFO:teuthology.orchestra.run.smithi082.stdout:(2/4): kernel-core-5.14.0-420.el9.x86_64.rpm 19 MB/s | 21 MB 00:01 2024-02-14T17:04:54.187 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-14T17:04:54.187 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-14T17:04:54.271 INFO:teuthology.orchestra.run.smithi082.stdout:(3/4): kernel-modules-5.14.0-420.el9.x86_64.rpm 15 MB/s | 39 MB 00:02 2024-02-14T17:04:54.280 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-02-14T17:04:54.291 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-02-14T17:04:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-02-14T17:04:54.291 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: kernel x86_64 5.14.0-420.el9 CentOS-BaseOS 6.0 M 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core x86_64 5.14.0-420.el9 CentOS-BaseOS 21 M 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules x86_64 5.14.0-420.el9 CentOS-BaseOS 39 M 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core x86_64 5.14.0-420.el9 CentOS-BaseOS 33 M 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout:Install 4 Packages 2024-02-14T17:04:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:04:54.293 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 99 M 2024-02-14T17:04:54.293 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 122 M 2024-02-14T17:04:54.293 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-02-14T17:04:54.563 INFO:teuthology.orchestra.run.smithi082.stdout:(4/4): kernel-modules-core-5.14.0-420.el9.x86_6 14 MB/s | 33 MB 00:02 2024-02-14T17:04:54.567 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:04:54.567 INFO:teuthology.orchestra.run.smithi082.stdout:Total 35 MB/s | 99 MB 00:02 2024-02-14T17:04:54.567 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-02-14T17:04:54.890 INFO:teuthology.orchestra.run.smithi120.stdout:(1/4): kernel-5.14.0-420.el9.x86_64.rpm 10 MB/s | 6.0 MB 00:00 2024-02-14T17:04:54.942 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-02-14T17:04:54.942 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-02-14T17:04:55.315 INFO:teuthology.orchestra.run.smithi120.stdout:(2/4): kernel-core-5.14.0-420.el9.x86_64.rpm 20 MB/s | 21 MB 00:01 2024-02-14T17:04:55.942 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-02-14T17:04:55.943 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-02-14T17:04:56.183 INFO:teuthology.orchestra.run.smithi120.stdout:(3/4): kernel-modules-5.14.0-420.el9.x86_64.rpm 21 MB/s | 39 MB 00:01 2024-02-14T17:04:56.816 INFO:teuthology.orchestra.run.smithi120.stdout:(4/4): kernel-modules-core-5.14.0-420.el9.x86_6 17 MB/s | 33 MB 00:01 2024-02-14T17:04:56.819 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:04:56.819 INFO:teuthology.orchestra.run.smithi120.stdout:Total 39 MB/s | 99 MB 00:02 2024-02-14T17:04:56.820 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-02-14T17:04:57.237 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-02-14T17:04:57.237 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-02-14T17:04:58.273 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-02-14T17:04:58.274 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-02-14T17:04:58.324 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-02-14T17:04:58.968 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-modules-core-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:04:59.097 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:05:00.707 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-02-14T17:05:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-core-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:05:01.660 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:05:01.836 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:05:02.017 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:05:04.470 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:05:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:05:06.712 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:05:06.820 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:09.353 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:05:09.463 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:10.486 INFO:teuthology.orchestra.run.smithi091.stdout:Extra Packages for Enterprise Linux 705 kB/s | 20 MB 00:29 2024-02-14T17:05:11.573 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:15.349 INFO:teuthology.orchestra.run.smithi091.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-02-14T17:05:17.564 INFO:teuthology.orchestra.run.smithi091.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-14T17:05:17.564 INFO:teuthology.orchestra.run.smithi091.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-14T17:05:17.658 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies resolved. 2024-02-14T17:05:17.667 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2024-02-14T17:05:17.667 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout: kernel x86_64 5.14.0-420.el9 CentOS-BaseOS 6.0 M 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout:Installing dependencies: 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout: kernel-core x86_64 5.14.0-420.el9 CentOS-BaseOS 21 M 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout: kernel-modules x86_64 5.14.0-420.el9 CentOS-BaseOS 39 M 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout: kernel-modules-core x86_64 5.14.0-420.el9 CentOS-BaseOS 33 M 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout:Install 4 Packages 2024-02-14T17:05:17.668 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:05:17.669 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 99 M 2024-02-14T17:05:17.669 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 122 M 2024-02-14T17:05:17.669 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading Packages: 2024-02-14T17:05:18.175 INFO:teuthology.orchestra.run.smithi091.stdout:(1/4): kernel-5.14.0-420.el9.x86_64.rpm 12 MB/s | 6.0 MB 00:00 2024-02-14T17:05:20.383 INFO:teuthology.orchestra.run.smithi091.stdout:(2/4): kernel-modules-5.14.0-420.el9.x86_64.rpm 14 MB/s | 39 MB 00:02 2024-02-14T17:05:20.584 INFO:teuthology.orchestra.run.smithi091.stdout:(3/4): kernel-core-5.14.0-420.el9.x86_64.rpm 7.2 MB/s | 21 MB 00:02 2024-02-14T17:05:21.459 INFO:teuthology.orchestra.run.smithi091.stdout:(4/4): kernel-modules-core-5.14.0-420.el9.x86_6 10 MB/s | 33 MB 00:03 2024-02-14T17:05:21.461 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:05:21.462 INFO:teuthology.orchestra.run.smithi091.stdout:Total 26 MB/s | 99 MB 00:03 2024-02-14T17:05:21.462 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2024-02-14T17:05:21.841 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction check succeeded. 2024-02-14T17:05:21.841 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2024-02-14T17:05:22.884 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded. 2024-02-14T17:05:22.885 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2024-02-14T17:05:25.295 INFO:teuthology.orchestra.run.smithi091.stdout: Preparing : 1/1 2024-02-14T17:05:25.949 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : kernel-modules-core-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:05:26.083 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:05:28.824 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:05:28.995 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:05:33.694 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:05:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:38.606 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:40.721 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:40.721 INFO:teuthology.orchestra.run.smithi082.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-14T17:05:40.721 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:05:40.791 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:05:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:05:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:05:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:05:41.721 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-02-14T17:05:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-5.14.0-420.el9.x86_64 2024-02-14T17:05:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-core-5.14.0-420.el9.x86_64 2024-02-14T17:05:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules-5.14.0-420.el9.x86_64 2024-02-14T17:05:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules-core-5.14.0-420.el9.x86_64 2024-02-14T17:05:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:05:41.722 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-02-14T17:05:42.238 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:05:42.366 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:05:42.366 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:05:42.367 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-420.el9.x86_64 2024-02-14T17:05:42.367 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-14T17:05:42.367 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-14T17:05:42.367 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-14T17:05:42.367 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi082.front.sepia.ceph.com, path=None, version=distro) 2024-02-14T17:05:42.367 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:05:42.475 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:05:42.475 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:05:42.475 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -qi grub2-tools 2024-02-14T17:05:42.508 INFO:teuthology.orchestra.run.smithi082.stdout:Name : grub2-tools 2024-02-14T17:05:42.508 INFO:teuthology.orchestra.run.smithi082.stdout:Epoch : 1 2024-02-14T17:05:42.508 INFO:teuthology.orchestra.run.smithi082.stdout:Version : 2.06 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Release : 68.el9 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Architecture: x86_64 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Group : Unspecified 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Size : 8264254 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:License : GPLv3+ 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Packager : builder@centos.org 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Vendor : CentOS 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-14T17:05:42.509 INFO:teuthology.orchestra.run.smithi082.stdout:Summary : Support tools for GRUB. 2024-02-14T17:05:42.510 INFO:teuthology.orchestra.run.smithi082.stdout:Description : 2024-02-14T17:05:42.510 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:05:42.510 INFO:teuthology.orchestra.run.smithi082.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-14T17:05:42.510 INFO:teuthology.orchestra.run.smithi082.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-14T17:05:42.510 INFO:teuthology.orchestra.run.smithi082.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-14T17:05:42.510 INFO:teuthology.orchestra.run.smithi082.stdout:hardware devices. 2024-02-14T17:05:42.510 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:05:42.510 INFO:teuthology.orchestra.run.smithi082.stdout:This subpackage provides tools for support of all platforms. 2024-02-14T17:05:42.511 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-14T17:05:42.511 INFO:teuthology.task.kernel:Updating grub on smithi082 to boot 5.14.0-420.el9.x86_64 2024-02-14T17:05:42.511 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-14T17:05:43.205 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2024-02-14T17:05:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:43.535 INFO:teuthology.orchestra.run.smithi120.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-14T17:05:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:05:43.605 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:05:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:05:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:05:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:05:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:05:44.423 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-02-14T17:05:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-5.14.0-420.el9.x86_64 2024-02-14T17:05:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core-5.14.0-420.el9.x86_64 2024-02-14T17:05:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-5.14.0-420.el9.x86_64 2024-02-14T17:05:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core-5.14.0-420.el9.x86_64 2024-02-14T17:05:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:05:44.423 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-02-14T17:05:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-14T17:05:44.602 INFO:teuthology.orchestra.run.smithi082.stderr:done 2024-02-14T17:05:44.604 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-02-14T17:05:44.619 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.I7U3oBiIx9 2024-02-14T17:05:44.619 DEBUG:teuthology.orchestra.run.smithi082:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.I7U3oBiIx9 2024-02-14T17:05:44.688 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.I7U3oBiIx9 2024-02-14T17:05:44.827 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.I7U3oBiIx9 is 6KB 2024-02-14T17:05:44.875 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.I7U3oBiIx9 2024-02-14T17:05:44.889 DEBUG:teuthology.orchestra.run.smithi082:> sudo /bin/ls /boot/loader/entries || true 2024-02-14T17:05:44.955 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:05:44.964 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-14T17:05:44.964 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-14T17:05:44.964 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-14T17:05:44.964 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64.conf 2024-02-14T17:05:44.966 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64 2024-02-14T17:05:45.082 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:05:45.083 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:05:45.083 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-420.el9.x86_64 2024-02-14T17:05:45.083 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-14T17:05:45.083 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-14T17:05:45.083 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-02-14T17:05:45.083 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi120.front.sepia.ceph.com, path=None, version=distro) 2024-02-14T17:05:45.083 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:05:45.128 DEBUG:teuthology.orchestra.run.smithi082:> sudo shutdown -r now 2024-02-14T17:05:45.191 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:05:45.191 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:05:45.191 DEBUG:teuthology.orchestra.run.smithi120:> sudo rpm -qi grub2-tools 2024-02-14T17:05:45.232 INFO:teuthology.orchestra.run.smithi120.stdout:Name : grub2-tools 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Epoch : 1 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Version : 2.06 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Release : 68.el9 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: x86_64 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Group : Unspecified 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Size : 8264254 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:License : GPLv3+ 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-14T17:05:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:Packager : builder@centos.org 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout:Vendor : CentOS 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout:Summary : Support tools for GRUB. 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout:Description : 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout:hardware devices. 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:05:45.234 INFO:teuthology.orchestra.run.smithi120.stdout:This subpackage provides tools for support of all platforms. 2024-02-14T17:05:45.235 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-14T17:05:45.235 INFO:teuthology.task.kernel:Updating grub on smithi120 to boot 5.14.0-420.el9.x86_64 2024-02-14T17:05:45.235 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-14T17:05:45.887 INFO:teuthology.orchestra.run.smithi120.stderr:Generating grub configuration file ... 2024-02-14T17:05:47.259 INFO:teuthology.orchestra.run.smithi120.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-14T17:05:47.278 INFO:teuthology.orchestra.run.smithi120.stderr:done 2024-02-14T17:05:47.280 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-02-14T17:05:47.295 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.pRDHXPPdH2 2024-02-14T17:05:47.295 DEBUG:teuthology.orchestra.run.smithi120:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.pRDHXPPdH2 2024-02-14T17:05:47.364 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.pRDHXPPdH2 2024-02-14T17:05:47.506 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.pRDHXPPdH2 is 6KB 2024-02-14T17:05:47.557 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.pRDHXPPdH2 2024-02-14T17:05:47.571 DEBUG:teuthology.orchestra.run.smithi120:> sudo /bin/ls /boot/loader/entries || true 2024-02-14T17:05:47.636 INFO:teuthology.orchestra.run.smithi120.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-14T17:05:47.636 INFO:teuthology.orchestra.run.smithi120.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-14T17:05:47.636 INFO:teuthology.orchestra.run.smithi120.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-14T17:05:47.636 INFO:teuthology.orchestra.run.smithi120.stdout:15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64.conf 2024-02-14T17:05:47.637 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64 2024-02-14T17:05:47.794 DEBUG:teuthology.orchestra.run.smithi120:> sudo shutdown -r now 2024-02-14T17:06:07.918 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:06:07.919 INFO:teuthology.orchestra.run.smithi091.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-14T17:06:07.919 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:06:07.989 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:06:08.427 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:06:08.428 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : kernel-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:06:08.428 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:06:08.428 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:06:08.874 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:06:08.874 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:06:08.875 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2024-02-14T17:06:08.875 INFO:teuthology.orchestra.run.smithi091.stdout: kernel-5.14.0-420.el9.x86_64 2024-02-14T17:06:08.875 INFO:teuthology.orchestra.run.smithi091.stdout: kernel-core-5.14.0-420.el9.x86_64 2024-02-14T17:06:08.875 INFO:teuthology.orchestra.run.smithi091.stdout: kernel-modules-5.14.0-420.el9.x86_64 2024-02-14T17:06:08.875 INFO:teuthology.orchestra.run.smithi091.stdout: kernel-modules-core-5.14.0-420.el9.x86_64 2024-02-14T17:06:08.875 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:06:08.875 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2024-02-14T17:06:09.354 DEBUG:teuthology.orchestra.run.smithi091:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:06:09.505 INFO:teuthology.orchestra.run.smithi091.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:06:09.505 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:06:09.505 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-420.el9.x86_64 2024-02-14T17:06:09.505 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-14T17:06:09.505 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-14T17:06:09.505 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-14T17:06:09.506 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi091.front.sepia.ceph.com, path=None, version=distro) 2024-02-14T17:06:09.506 DEBUG:teuthology.orchestra.run.smithi091:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:06:09.612 INFO:teuthology.orchestra.run.smithi091.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:06:09.613 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:06:09.613 DEBUG:teuthology.orchestra.run.smithi091:> sudo rpm -qi grub2-tools 2024-02-14T17:06:09.655 INFO:teuthology.orchestra.run.smithi091.stdout:Name : grub2-tools 2024-02-14T17:06:09.655 INFO:teuthology.orchestra.run.smithi091.stdout:Epoch : 1 2024-02-14T17:06:09.655 INFO:teuthology.orchestra.run.smithi091.stdout:Version : 2.06 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:Release : 68.el9 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:Architecture: x86_64 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:Group : Unspecified 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:Size : 8264254 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:License : GPLv3+ 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-14T17:06:09.656 INFO:teuthology.orchestra.run.smithi091.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:Packager : builder@centos.org 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:Vendor : CentOS 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:Summary : Support tools for GRUB. 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:Description : 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:hardware devices. 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:06:09.657 INFO:teuthology.orchestra.run.smithi091.stdout:This subpackage provides tools for support of all platforms. 2024-02-14T17:06:09.658 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-14T17:06:09.658 INFO:teuthology.task.kernel:Updating grub on smithi091 to boot 5.14.0-420.el9.x86_64 2024-02-14T17:06:09.658 DEBUG:teuthology.orchestra.run.smithi091:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-14T17:06:10.214 INFO:teuthology.orchestra.run.smithi091.stderr:Generating grub configuration file ... 2024-02-14T17:06:11.628 INFO:teuthology.orchestra.run.smithi091.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-14T17:06:11.675 INFO:teuthology.orchestra.run.smithi091.stderr:done 2024-02-14T17:06:11.678 DEBUG:teuthology.orchestra.run.smithi091:> mktemp 2024-02-14T17:06:11.693 INFO:teuthology.orchestra.run.smithi091.stdout:/tmp/tmp.suSLhvcXVE 2024-02-14T17:06:11.693 DEBUG:teuthology.orchestra.run.smithi091:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.suSLhvcXVE 2024-02-14T17:06:11.762 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0666 /tmp/tmp.suSLhvcXVE 2024-02-14T17:06:11.903 DEBUG:teuthology.orchestra.remote:smithi091:/tmp/tmp.suSLhvcXVE is 6KB 2024-02-14T17:06:11.950 DEBUG:teuthology.orchestra.run.smithi091:> rm -fr /tmp/tmp.suSLhvcXVE 2024-02-14T17:06:11.964 DEBUG:teuthology.orchestra.run.smithi091:> sudo /bin/ls /boot/loader/entries || true 2024-02-14T17:06:12.029 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-14T17:06:12.029 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-14T17:06:12.029 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-14T17:06:12.029 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64.conf 2024-02-14T17:06:12.030 DEBUG:teuthology.orchestra.run.smithi091:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64 2024-02-14T17:06:12.180 DEBUG:teuthology.orchestra.run.smithi091:> sudo shutdown -r now 2024-02-14T17:06:15.133 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-14T17:06:15.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-14T17:06:15.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:06:17.800 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-02-14T17:06:17.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-02-14T17:06:17.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:06:33.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-02-14T17:06:36.220 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-02-14T17:06:42.188 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-14T17:06:42.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-14T17:06:42.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:06:42.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-14T17:06:42.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:06:45.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-02-14T17:06:45.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:06:45.724 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-02-14T17:06:48.284 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-02-14T17:06:57.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-14T17:06:57.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:07:00.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-02-14T17:07:00.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:07:00.540 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.91 2024-02-14T17:07:09.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-14T17:07:09.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:07:12.604 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.91 2024-02-14T17:07:18.716 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-02-14T17:07:24.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-14T17:07:24.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:07:29.533 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-02-14T17:07:29.972 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-14T17:07:29.972 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-420.el9.x86_64"... 2024-02-14T17:07:29.972 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-02-14T17:07:29.986 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-420.el9.x86_64 2024-02-14T17:07:29.987 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-420.el9.x86_64 vs 5.14.0-420.el9.x86_64 2024-02-14T17:07:29.987 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-14T17:07:29.987 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-14T17:07:30.987 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-14T17:07:30.988 DEBUG:teuthology.orchestra.run.smithi082:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-14T17:07:31.074 INFO:teuthology.orchestra.run.smithi082.stdout:ttyS1 2024-02-14T17:07:31.115 DEBUG:teuthology.parallel:result is None 2024-02-14T17:07:33.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-02-14T17:07:33.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:07:34.012 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-02-14T17:07:34.415 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-02-14T17:07:34.415 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-420.el9.x86_64"... 2024-02-14T17:07:34.415 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-02-14T17:07:34.429 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-420.el9.x86_64 2024-02-14T17:07:34.429 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-420.el9.x86_64 vs 5.14.0-420.el9.x86_64 2024-02-14T17:07:34.429 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-14T17:07:34.429 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-14T17:07:35.431 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-02-14T17:07:35.431 DEBUG:teuthology.orchestra.run.smithi120:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-14T17:07:35.499 INFO:teuthology.orchestra.run.smithi120.stdout:ttyS1 2024-02-14T17:07:35.544 DEBUG:teuthology.parallel:result is None 2024-02-14T17:07:43.036 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.91 2024-02-14T17:07:58.037 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-14T17:07:58.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:07:58.485 DEBUG:teuthology.orchestra.run.smithi091:> true 2024-02-14T17:07:58.904 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-14T17:07:58.905 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-420.el9.x86_64"... 2024-02-14T17:07:58.905 DEBUG:teuthology.orchestra.run.smithi091:> uname -r 2024-02-14T17:07:58.919 INFO:teuthology.orchestra.run.smithi091.stdout:5.14.0-420.el9.x86_64 2024-02-14T17:07:58.919 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-420.el9.x86_64 vs 5.14.0-420.el9.x86_64 2024-02-14T17:07:58.919 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-14T17:07:58.919 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-14T17:07:59.920 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-14T17:07:59.920 DEBUG:teuthology.orchestra.run.smithi091:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-14T17:08:00.028 INFO:teuthology.orchestra.run.smithi091.stdout:ttyS1 2024-02-14T17:08:00.066 DEBUG:teuthology.parallel:result is None 2024-02-14T17:08:00.067 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-14T17:08:00.074 INFO:teuthology.task.internal:Creating test directory... 2024-02-14T17:08:00.075 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T17:08:00.078 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T17:08:00.080 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T17:08:00.098 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-14T17:08:00.104 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-14T17:08:00.115 INFO:teuthology.task.internal:Creating archive directory... 2024-02-14T17:08:00.115 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T17:08:00.138 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T17:08:00.142 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T17:08:00.176 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-14T17:08:00.183 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-14T17:08:00.183 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T17:08:00.202 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T17:08:00.204 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T17:08:00.237 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:08:00.244 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:08:00.251 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:08:00.251 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:08:00.258 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:08:00.264 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:08:00.265 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-14T17:08:00.275 INFO:teuthology.task.internal:Configuring sudo... 2024-02-14T17:08:00.275 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T17:08:00.296 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T17:08:00.298 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T17:08:00.329 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-14T17:08:00.341 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-14T17:08:00.341 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T17:08:00.368 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T17:08:00.370 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T17:08:00.386 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:08:00.468 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:08:00.518 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:08:00.601 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:08:00.642 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:08:00.643 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T17:08:00.710 DEBUG:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:08:00.751 DEBUG:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:08:00.811 DEBUG:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:08:00.895 DEBUG:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:08:00.941 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:08:00.941 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T17:08:01.011 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:08:01.060 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:08:01.112 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:08:01.197 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:08:01.243 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:08:01.243 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T17:08:01.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-02-14T17:08:01.315 DEBUG:teuthology.orchestra.run.smithi091:> sudo service rsyslog restart 2024-02-14T17:08:01.317 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-02-14T17:08:01.375 INFO:teuthology.orchestra.run.smithi091.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:08:01.377 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:08:01.401 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:08:01.843 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-14T17:08:01.854 INFO:teuthology.task.internal:Starting timer... 2024-02-14T17:08:01.855 INFO:teuthology.run_tasks:Running task pcp... 2024-02-14T17:08:01.889 INFO:teuthology.run_tasks:Running task selinux... 2024-02-14T17:08:01.939 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-14T17:08:01.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-02-14T17:08:01.987 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-02-14T17:08:01.989 DEBUG:teuthology.orchestra.run.smithi091:> sudo service auditd rotate 2024-02-14T17:08:02.048 INFO:teuthology.orchestra.run.smithi091.stdout:Rotating logs: 2024-02-14T17:08:02.049 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-02-14T17:08:02.114 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-02-14T17:08:02.116 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-14T17:08:02.116 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-02-14T17:08:02.136 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-02-14T17:08:02.136 DEBUG:teuthology.orchestra.run.smithi091:> /usr/sbin/getenforce 2024-02-14T17:08:02.164 INFO:teuthology.orchestra.run.smithi091.stdout:Permissive 2024-02-14T17:08:02.165 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-02-14T17:08:02.186 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-02-14T17:08:02.187 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi091.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi120.front.sepia.ceph.com': 'permissive'} 2024-02-14T17:08:02.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:08:02.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:08:02.216 DEBUG:teuthology.orchestra.run.smithi091:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:08:02.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:08:02.254 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:08:02.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:08:02.283 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-14T17:08:02.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-02-14T17:08:02.321 DEBUG:teuthology.orchestra.run.smithi091:> sudo /usr/sbin/setenforce permissive 2024-02-14T17:08:02.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-02-14T17:08:02.388 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-14T17:08:02.398 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-14T17:08:02.414 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-14T17:08:02.415 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com,smithi091.front.sepia.ceph.com,smithi120.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-14T17:12:17.758 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi091.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com')] 2024-02-14T17:12:17.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-14T17:12:17.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:12:17.852 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-02-14T17:12:17.938 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-14T17:12:17.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-14T17:12:17.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:12:18.028 DEBUG:teuthology.orchestra.run.smithi091:> true 2024-02-14T17:12:18.102 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-14T17:12:18.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-02-14T17:12:18.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:12:18.196 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-02-14T17:12:18.275 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-02-14T17:12:18.275 INFO:teuthology.run_tasks:Running task clock... 2024-02-14T17:12:18.287 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-14T17:12:18.287 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T17:12:18.289 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:12:18.291 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T17:12:18.291 DEBUG:teuthology.orchestra.run.smithi091:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:12:18.293 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T17:12:18.294 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:12:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T17:12:18.321 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T17:12:18.337 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T17:12:18.338 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T17:12:18.347 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T17:12:18.364 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T17:12:18.365 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-02-14T17:12:18.367 INFO:teuthology.orchestra.run.smithi091.stderr:sudo: ntpd: command not found 2024-02-14T17:12:18.379 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-02-14T17:12:18.381 INFO:teuthology.orchestra.run.smithi091.stdout:506 Cannot talk to daemon 2024-02-14T17:12:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-02-14T17:12:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T17:12:18.398 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T17:12:18.406 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-02-14T17:12:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T17:12:18.413 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T17:12:18.422 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T17:12:18.438 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T17:12:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:12:18.468 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:12:18.468 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-02-14T17:12:18.468 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.468 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.468 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.468 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.471 INFO:teuthology.orchestra.run.smithi091.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:12:18.472 INFO:teuthology.orchestra.run.smithi091.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:12:18.472 INFO:teuthology.orchestra.run.smithi091.stdout:=============================================================================== 2024-02-14T17:12:18.473 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.473 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.473 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.473 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.492 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:12:18.495 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:12:18.495 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-02-14T17:12:18.495 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.495 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.495 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.495 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:12:18.496 INFO:teuthology.run_tasks:Running task pexec... 2024-02-14T17:12:18.510 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-14T17:12:18.510 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi082.front.sepia.ceph.com 2024-02-14T17:12:18.510 DEBUG:teuthology.orchestra.run.smithi082:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-14T17:12:18.511 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi091.front.sepia.ceph.com 2024-02-14T17:12:18.511 DEBUG:teuthology.orchestra.run.smithi091:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-14T17:12:18.511 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi120.front.sepia.ceph.com 2024-02-14T17:12:18.511 DEBUG:teuthology.orchestra.run.smithi120:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-14T17:12:19.104 INFO:teuthology.orchestra.run.smithi091.stdout:Last metadata expiration check: 0:00:50 ago on Wed 14 Feb 2024 05:11:28 PM UTC. 2024-02-14T17:12:19.108 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:50 ago on Wed 14 Feb 2024 05:11:28 PM UTC. 2024-02-14T17:12:19.129 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:00:50 ago on Wed 14 Feb 2024 05:11:29 PM UTC. 2024-02-14T17:12:19.263 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies resolved. 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout: Package Architecture Version Repository Size 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package 2024-02-14T17:12:19.265 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:12:19.266 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 3.1 M 2024-02-14T17:12:19.266 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 9.8 M 2024-02-14T17:12:19.266 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading Packages: 2024-02-14T17:12:19.268 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-02-14T17:12:19.285 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-02-14T17:12:19.285 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2024-02-14T17:12:19.285 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-02-14T17:12:19.285 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-02-14T17:12:19.285 INFO:teuthology.orchestra.run.smithi082.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-14T17:12:19.285 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:12:19.286 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-02-14T17:12:19.286 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-02-14T17:12:19.286 INFO:teuthology.orchestra.run.smithi082.stdout:Install 1 Package 2024-02-14T17:12:19.286 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:12:19.286 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 3.1 M 2024-02-14T17:12:19.286 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 9.8 M 2024-02-14T17:12:19.286 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-02-14T17:12:19.291 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-02-14T17:12:19.304 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-02-14T17:12:19.304 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-02-14T17:12:19.304 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-02-14T17:12:19.304 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-02-14T17:12:19.304 INFO:teuthology.orchestra.run.smithi120.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-14T17:12:19.304 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:12:19.304 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-02-14T17:12:19.305 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-02-14T17:12:19.305 INFO:teuthology.orchestra.run.smithi120.stdout:Install 1 Package 2024-02-14T17:12:19.305 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:12:19.305 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 3.1 M 2024-02-14T17:12:19.305 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 9.8 M 2024-02-14T17:12:19.305 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-02-14T17:12:19.465 INFO:teuthology.orchestra.run.smithi091.stdout:runc-1.1.12-1.el9.x86_64.rpm 16 MB/s | 3.1 MB 00:00 2024-02-14T17:12:19.466 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:12:19.466 INFO:teuthology.orchestra.run.smithi091.stdout:Total 16 MB/s | 3.1 MB 00:00 2024-02-14T17:12:19.466 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2024-02-14T17:12:19.471 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction check succeeded. 2024-02-14T17:12:19.471 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2024-02-14T17:12:19.529 INFO:teuthology.orchestra.run.smithi082.stdout:runc-1.1.12-1.el9.x86_64.rpm 13 MB/s | 3.1 MB 00:00 2024-02-14T17:12:19.529 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:12:19.530 INFO:teuthology.orchestra.run.smithi082.stdout:Total 13 MB/s | 3.1 MB 00:00 2024-02-14T17:12:19.530 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-02-14T17:12:19.535 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-02-14T17:12:19.535 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-02-14T17:12:19.542 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded. 2024-02-14T17:12:19.542 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2024-02-14T17:12:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:runc-1.1.12-1.el9.x86_64.rpm 12 MB/s | 3.1 MB 00:00 2024-02-14T17:12:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:12:19.568 INFO:teuthology.orchestra.run.smithi120.stdout:Total 12 MB/s | 3.1 MB 00:00 2024-02-14T17:12:19.568 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-02-14T17:12:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-02-14T17:12:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-02-14T17:12:19.607 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-02-14T17:12:19.607 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-02-14T17:12:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-02-14T17:12:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-02-14T17:12:19.751 INFO:teuthology.orchestra.run.smithi091.stdout: Preparing : 1/1 2024-02-14T17:12:19.803 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-02-14T17:12:19.826 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:12:19.849 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-02-14T17:12:19.890 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:12:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:12:20.393 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:12:20.479 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:12:20.736 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:12:20.736 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:12:20.736 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2024-02-14T17:12:20.736 INFO:teuthology.orchestra.run.smithi091.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-14T17:12:20.736 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-14T17:12:20.736 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2024-02-14T17:12:20.767 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:12:20.767 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:12:20.767 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-02-14T17:12:20.767 INFO:teuthology.orchestra.run.smithi082.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-14T17:12:20.767 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:12:20.767 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-02-14T17:12:20.876 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:12:20.876 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:12:20.876 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-02-14T17:12:20.876 INFO:teuthology.orchestra.run.smithi120.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-14T17:12:20.876 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:12:20.876 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-02-14T17:12:20.993 DEBUG:teuthology.parallel:result is None 2024-02-14T17:12:21.011 DEBUG:teuthology.parallel:result is None 2024-02-14T17:12:21.160 DEBUG:teuthology.parallel:result is None 2024-02-14T17:12:21.160 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-02-14T17:12:21.171 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-02-14T17:12:21.171 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:12:21.171 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:12:21.188 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T17:12:21.189 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-02-14T17:12:21.243 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T17:12:21.243 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:21.243 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-14T17:12:21.243 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:21.243 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:21.243 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-14 17:12:16.761445477 +0000 2024-02-14T17:12:21.243 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-14 17:11:54.197881933 +0000 2024-02-14T17:12:21.243 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-14 17:11:54.197881933 +0000 2024-02-14T17:12:21.243 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-14 17:11:54.197881933 +0000 2024-02-14T17:12:21.244 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T17:12:21.309 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-14T17:12:21.309 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-14T17:12:21.309 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000212573 s, 2.4 MB/s 2024-02-14T17:12:21.310 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T17:12:21.366 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-02-14T17:12:21.421 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T17:12:21.421 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:21.421 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-14T17:12:21.422 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:21.422 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:21.422 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-14 17:12:16.761445477 +0000 2024-02-14T17:12:21.422 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-14 17:11:54.538875337 +0000 2024-02-14T17:12:21.422 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-14 17:11:54.538875337 +0000 2024-02-14T17:12:21.422 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-14 17:11:54.538875337 +0000 2024-02-14T17:12:21.423 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T17:12:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-14T17:12:21.487 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-14T17:12:21.487 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000187831 s, 2.7 MB/s 2024-02-14T17:12:21.488 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T17:12:21.543 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-02-14T17:12:21.598 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T17:12:21.598 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:21.598 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-14T17:12:21.598 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:21.598 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:21.598 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-14 17:12:16.761445477 +0000 2024-02-14T17:12:21.598 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-14 17:11:54.866868992 +0000 2024-02-14T17:12:21.598 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-14 17:11:54.866868992 +0000 2024-02-14T17:12:21.598 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-14 17:11:54.866868992 +0000 2024-02-14T17:12:21.599 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T17:12:21.664 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-14T17:12:21.664 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-14T17:12:21.664 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000193297 s, 2.6 MB/s 2024-02-14T17:12:21.665 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T17:12:21.721 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-02-14T17:12:21.776 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T17:12:21.776 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:21.776 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-14T17:12:21.776 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:21.776 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:21.776 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-14 17:12:16.761445477 +0000 2024-02-14T17:12:21.777 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-14 17:11:55.209862358 +0000 2024-02-14T17:12:21.777 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-14 17:11:55.209862358 +0000 2024-02-14T17:12:21.777 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-14 17:11:55.209862358 +0000 2024-02-14T17:12:21.777 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T17:12:21.842 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-14T17:12:21.842 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-14T17:12:21.842 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.00023711 s, 2.2 MB/s 2024-02-14T17:12:21.843 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T17:12:21.899 DEBUG:teuthology.orchestra.run.smithi082:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-14T17:12:22.063 INFO:teuthology.orchestra.run.smithi082.stdout:loop 2024-02-14T17:12:22.064 INFO:tasks.nvme_loop:Connecting nvme_loop smithi082:/dev/vg_nvme/lv_1... 2024-02-14T17:12:22.065 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-14T17:12:22.102 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-14T17:12:22.139 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/vg_nvme/lv_11 2024-02-14T17:12:22.176 INFO:tasks.nvme_loop:Connecting nvme_loop smithi082:/dev/vg_nvme/lv_2... 2024-02-14T17:12:22.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-14T17:12:22.266 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-14T17:12:22.304 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/vg_nvme/lv_21 2024-02-14T17:12:22.334 INFO:tasks.nvme_loop:Connecting nvme_loop smithi082:/dev/vg_nvme/lv_3... 2024-02-14T17:12:22.334 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-14T17:12:22.380 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-14T17:12:22.419 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/vg_nvme/lv_31 2024-02-14T17:12:22.448 INFO:tasks.nvme_loop:Connecting nvme_loop smithi082:/dev/vg_nvme/lv_4... 2024-02-14T17:12:22.449 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-14T17:12:22.527 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-14T17:12:22.566 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/vg_nvme/lv_41 2024-02-14T17:12:22.596 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:12:22.596 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:12:22.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo nvme list 2024-02-14T17:12:22.717 INFO:teuthology.orchestra.run.smithi082.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-14T17:12:22.717 INFO:teuthology.orchestra.run.smithi082.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-14T17:12:22.717 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/nvme0n1 PHFT620400SD400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-14T17:12:22.717 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/nvme1n1 4f764867856cc8ce0cd6 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:22.717 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/nvme2n1 364aff1b9151f94e5324 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:22.717 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/nvme3n1 8618400abe87701ac7cc Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:22.717 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/nvme4n1 6a816038c081d050ccd8 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:22.718 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-14T17:12:22.719 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:12:22.719 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/scratch_devs 2024-02-14T17:12:22.786 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:12:22.787 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:12:22.805 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T17:12:22.805 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_1 2024-02-14T17:12:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T17:12:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:22.863 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-14T17:12:22.863 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:22.863 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:22.863 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-14 17:12:16.782962864 +0000 2024-02-14T17:12:22.863 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-14 17:11:54.953517364 +0000 2024-02-14T17:12:22.863 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-14 17:11:54.953517364 +0000 2024-02-14T17:12:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-14 17:11:54.953517364 +0000 2024-02-14T17:12:22.863 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T17:12:22.931 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-14T17:12:22.931 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-14T17:12:22.932 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.00027456 s, 1.9 MB/s 2024-02-14T17:12:22.933 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T17:12:22.990 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_2 2024-02-14T17:12:23.046 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T17:12:23.046 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:23.046 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-14T17:12:23.046 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:23.046 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:23.046 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-14 17:12:16.782962864 +0000 2024-02-14T17:12:23.047 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-14 17:11:55.303508474 +0000 2024-02-14T17:12:23.047 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-14 17:11:55.303508474 +0000 2024-02-14T17:12:23.047 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-14 17:11:55.303508474 +0000 2024-02-14T17:12:23.047 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T17:12:23.113 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-14T17:12:23.114 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-14T17:12:23.114 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000314854 s, 1.6 MB/s 2024-02-14T17:12:23.115 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T17:12:23.171 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_3 2024-02-14T17:12:23.226 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T17:12:23.226 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:23.226 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-14T17:12:23.226 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:23.226 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:23.226 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-14 17:12:16.782962864 +0000 2024-02-14T17:12:23.226 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-14 17:11:55.637499990 +0000 2024-02-14T17:12:23.226 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-14 17:11:55.637499990 +0000 2024-02-14T17:12:23.226 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-14 17:11:55.637499990 +0000 2024-02-14T17:12:23.227 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T17:12:23.292 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-14T17:12:23.292 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-14T17:12:23.292 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.00018582 s, 2.8 MB/s 2024-02-14T17:12:23.293 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T17:12:23.349 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_4 2024-02-14T17:12:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T17:12:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:23.404 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-14T17:12:23.404 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:23.404 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:23.404 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-14 17:12:16.782962864 +0000 2024-02-14T17:12:23.404 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-14 17:11:55.965491658 +0000 2024-02-14T17:12:23.405 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-14 17:11:55.965491658 +0000 2024-02-14T17:12:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-14 17:11:55.965491658 +0000 2024-02-14T17:12:23.405 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T17:12:23.469 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-14T17:12:23.470 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-14T17:12:23.470 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000202349 s, 2.5 MB/s 2024-02-14T17:12:23.471 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T17:12:23.527 DEBUG:teuthology.orchestra.run.smithi091:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-14T17:12:23.669 INFO:teuthology.orchestra.run.smithi091.stdout:loop 2024-02-14T17:12:23.670 INFO:tasks.nvme_loop:Connecting nvme_loop smithi091:/dev/vg_nvme/lv_1... 2024-02-14T17:12:23.670 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-14T17:12:23.708 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-14T17:12:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/vg_nvme/lv_11 2024-02-14T17:12:23.781 INFO:tasks.nvme_loop:Connecting nvme_loop smithi091:/dev/vg_nvme/lv_2... 2024-02-14T17:12:23.781 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-14T17:12:23.863 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-14T17:12:23.901 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/vg_nvme/lv_21 2024-02-14T17:12:23.938 INFO:tasks.nvme_loop:Connecting nvme_loop smithi091:/dev/vg_nvme/lv_3... 2024-02-14T17:12:23.938 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-14T17:12:24.017 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-14T17:12:24.055 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/vg_nvme/lv_31 2024-02-14T17:12:24.085 INFO:tasks.nvme_loop:Connecting nvme_loop smithi091:/dev/vg_nvme/lv_4... 2024-02-14T17:12:24.085 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-14T17:12:24.165 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-14T17:12:24.204 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/vg_nvme/lv_41 2024-02-14T17:12:24.234 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:12:24.234 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:12:24.289 DEBUG:teuthology.orchestra.run.smithi091:> sudo nvme list 2024-02-14T17:12:24.355 INFO:teuthology.orchestra.run.smithi091.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-14T17:12:24.356 INFO:teuthology.orchestra.run.smithi091.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-14T17:12:24.356 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/nvme0n1 CVFT623300AG400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-14T17:12:24.356 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/nvme1n1 6bcc575aa7cf1969d39d Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:24.356 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/nvme2n1 cf342decd805f61a2b68 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:24.356 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/nvme3n1 2237913d6ea39ee4b780 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:24.356 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/nvme4n1 deadadc57fa37af37191 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:24.357 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-14T17:12:24.357 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:12:24.357 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/scratch_devs 2024-02-14T17:12:24.424 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:12:24.424 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:12:24.442 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T17:12:24.443 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-02-14T17:12:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T17:12:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:24.501 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-14T17:12:24.501 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:24.501 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:24.501 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-02-14 17:12:16.738677035 +0000 2024-02-14T17:12:24.501 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-02-14 17:11:54.609188277 +0000 2024-02-14T17:12:24.501 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-02-14 17:11:54.609188277 +0000 2024-02-14T17:12:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-02-14 17:11:54.609188277 +0000 2024-02-14T17:12:24.501 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T17:12:24.570 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-02-14T17:12:24.570 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-02-14T17:12:24.570 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000215746 s, 2.4 MB/s 2024-02-14T17:12:24.571 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T17:12:24.629 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-02-14T17:12:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T17:12:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:24.685 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-14T17:12:24.685 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:24.685 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:24.685 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-02-14 17:12:16.738677035 +0000 2024-02-14T17:12:24.685 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-02-14 17:11:54.949180442 +0000 2024-02-14T17:12:24.686 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-02-14 17:11:54.949180442 +0000 2024-02-14T17:12:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-02-14 17:11:54.949180442 +0000 2024-02-14T17:12:24.686 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T17:12:24.752 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-02-14T17:12:24.752 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-02-14T17:12:24.752 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000196487 s, 2.6 MB/s 2024-02-14T17:12:24.753 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T17:12:24.808 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-02-14T17:12:24.864 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T17:12:24.864 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:24.864 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-14T17:12:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-02-14 17:12:16.738677035 +0000 2024-02-14T17:12:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-02-14 17:11:55.299172356 +0000 2024-02-14T17:12:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-02-14 17:11:55.299172356 +0000 2024-02-14T17:12:24.865 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-02-14 17:11:55.299172356 +0000 2024-02-14T17:12:24.865 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T17:12:24.931 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-02-14T17:12:24.931 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-02-14T17:12:24.931 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000185422 s, 2.8 MB/s 2024-02-14T17:12:24.932 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T17:12:24.987 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-02-14T17:12:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T17:12:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:12:25.043 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-14T17:12:25.043 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:12:25.043 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:12:25.043 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-02-14 17:12:16.738677035 +0000 2024-02-14T17:12:25.043 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-02-14 17:11:55.622164894 +0000 2024-02-14T17:12:25.044 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-02-14 17:11:55.622164894 +0000 2024-02-14T17:12:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-02-14 17:11:55.622164894 +0000 2024-02-14T17:12:25.044 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T17:12:25.108 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-02-14T17:12:25.108 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-02-14T17:12:25.108 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000208024 s, 2.5 MB/s 2024-02-14T17:12:25.110 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T17:12:25.165 DEBUG:teuthology.orchestra.run.smithi120:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-14T17:12:25.313 INFO:teuthology.orchestra.run.smithi120.stdout:loop 2024-02-14T17:12:25.314 INFO:tasks.nvme_loop:Connecting nvme_loop smithi120:/dev/vg_nvme/lv_1... 2024-02-14T17:12:25.314 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-14T17:12:25.351 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-02-14T17:12:25.388 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/vg_nvme/lv_11 2024-02-14T17:12:25.417 INFO:tasks.nvme_loop:Connecting nvme_loop smithi120:/dev/vg_nvme/lv_2... 2024-02-14T17:12:25.418 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-14T17:12:25.508 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-02-14T17:12:25.547 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/vg_nvme/lv_21 2024-02-14T17:12:25.584 INFO:tasks.nvme_loop:Connecting nvme_loop smithi120:/dev/vg_nvme/lv_3... 2024-02-14T17:12:25.584 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-14T17:12:25.624 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-02-14T17:12:25.662 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/vg_nvme/lv_31 2024-02-14T17:12:25.698 INFO:tasks.nvme_loop:Connecting nvme_loop smithi120:/dev/vg_nvme/lv_4... 2024-02-14T17:12:25.698 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-14T17:12:25.776 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-02-14T17:12:25.815 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/vg_nvme/lv_41 2024-02-14T17:12:25.852 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:12:25.853 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:12:25.909 DEBUG:teuthology.orchestra.run.smithi120:> sudo nvme list 2024-02-14T17:12:25.975 INFO:teuthology.orchestra.run.smithi120.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-14T17:12:25.975 INFO:teuthology.orchestra.run.smithi120.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-14T17:12:25.975 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/nvme0n1 PHFT620400XV400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101F0 2024-02-14T17:12:25.975 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/nvme1n1 c7437a01a5cab3a8df82 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:25.975 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/nvme2n1 5175f937f3136689a646 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:25.975 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/nvme3n1 de5d6bd4677e9abf306b Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:25.975 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/nvme4n1 08f82a3d3cdd0c3b1bd2 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-14T17:12:25.976 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-14T17:12:25.977 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:12:25.977 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/scratch_devs 2024-02-14T17:12:26.044 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-14T17:12:26.166 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1'} 2024-02-14T17:12:26.167 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:12:26.168 INFO:tasks.cephadm:Cluster fsid is 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:12:26.168 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-14T17:12:26.168 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.82', 'mon.b': '172.21.15.91', 'mon.c': '172.21.15.120'} 2024-02-14T17:12:26.168 INFO:tasks.cephadm:First mon is mon.a on smithi082 2024-02-14T17:12:26.168 INFO:tasks.cephadm:First mgr is y 2024-02-14T17:12:26.169 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-14T17:12:26.169 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2024-02-14T17:12:26.202 DEBUG:teuthology.orchestra.run.smithi091:> sudo hostname $(hostname -s) 2024-02-14T17:12:26.235 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname $(hostname -s) 2024-02-14T17:12:26.262 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-14T17:12:26.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:12:26.404 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76734/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1430-g8727ce93', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1430.g8727ce93'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-14 13:20:25.883975', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-testing-2024-02-14-1033', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-14T17:12:26.546 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-guits-testing-2024-02-14-1033, sha1 8727ce939fc044a98f215e29696bb1433f5758d1 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:12:26.548 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/x86_64/flavors/default/cephadm 2024-02-14T17:12:26.548 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-14T17:12:27.415 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r--. 1 ubuntu ubuntu 767024 Feb 14 17:12 /home/ubuntu/cephtest/cephadm 2024-02-14T17:12:27.416 DEBUG:teuthology.orchestra.run.smithi091:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-14T17:12:27.988 INFO:teuthology.orchestra.run.smithi091.stdout:-rw-r--r--. 1 ubuntu ubuntu 767024 Feb 14 17:12 /home/ubuntu/cephtest/cephadm 2024-02-14T17:12:27.988 DEBUG:teuthology.orchestra.run.smithi120:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-14T17:12:28.935 INFO:teuthology.orchestra.run.smithi120.stdout:-rw-r--r--. 1 ubuntu ubuntu 767024 Feb 14 17:12 /home/ubuntu/cephtest/cephadm 2024-02-14T17:12:28.936 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-14T17:12:28.954 DEBUG:teuthology.orchestra.run.smithi091:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-14T17:12:28.973 DEBUG:teuthology.orchestra.run.smithi120:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-14T17:12:29.006 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 on all hosts... 2024-02-14T17:12:29.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 pull 2024-02-14T17:12:29.008 DEBUG:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 pull 2024-02-14T17:12:29.015 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 pull 2024-02-14T17:12:29.780 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:12:29.808 INFO:teuthology.orchestra.run.smithi091.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:12:29.810 INFO:teuthology.orchestra.run.smithi120.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:13:04.662 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-14T17:13:04.663 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:13:04.663 INFO:teuthology.orchestra.run.smithi091.stdout: "image_id": "170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86", 2024-02-14T17:13:04.663 INFO:teuthology.orchestra.run.smithi091.stdout: "repo_digests": [ 2024-02-14T17:13:04.663 INFO:teuthology.orchestra.run.smithi091.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b" 2024-02-14T17:13:04.663 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-14T17:13:04.663 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-14T17:13:12.096 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-02-14T17:13:12.097 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:13:12.097 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86", 2024-02-14T17:13:12.097 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2024-02-14T17:13:12.097 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b" 2024-02-14T17:13:12.097 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-02-14T17:13:12.097 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-02-14T17:13:18.798 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-02-14T17:13:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:13:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "image_id": "170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86", 2024-02-14T17:13:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "repo_digests": [ 2024-02-14T17:13:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b" 2024-02-14T17:13:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-02-14T17:13:18.799 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-02-14T17:13:18.822 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2024-02-14T17:13:18.859 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /etc/ceph 2024-02-14T17:13:18.899 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph 2024-02-14T17:13:18.928 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2024-02-14T17:13:18.959 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 777 /etc/ceph 2024-02-14T17:13:18.989 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /etc/ceph 2024-02-14T17:13:19.016 INFO:tasks.cephadm:Writing seed config... 2024-02-14T17:13:19.017 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-14T17:13:19.017 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-14T17:13:19.017 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-02-14T17:13:19.017 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-14T17:13:19.017 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-14T17:13:19.017 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-14T17:13:19.017 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-14T17:13:19.018 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-14T17:13:19.018 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-14T17:13:19.018 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-14T17:13:19.018 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:13:19.018 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-14T17:13:19.036 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 39c4996e-cb5c-11ee-95b9-87774f69a715 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-02-14T17:13:19.036 DEBUG:teuthology.orchestra.run.smithi082:mon.a> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.a.service 2024-02-14T17:13:19.079 DEBUG:teuthology.orchestra.run.smithi082:mgr.y> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.y.service 2024-02-14T17:13:19.122 INFO:tasks.cephadm:Bootstrapping... 2024-02-14T17:13:19.122 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 -v bootstrap --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.82 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:13:19.317 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:13:19.318 INFO:teuthology.orchestra.run.smithi082.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1', '-v', 'bootstrap', '--fsid', '39c4996e-cb5c-11ee-95b9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.82', '--skip-admin-label'] 2024-02-14T17:13:19.343 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T17:13:19.343 INFO:teuthology.orchestra.run.smithi082.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-14T17:13:19.344 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying podman|docker is present... 2024-02-14T17:13:19.366 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T17:13:19.366 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying lvm2 is present... 2024-02-14T17:13:19.366 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying time synchronization is in place... 2024-02-14T17:13:19.373 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-14T17:13:19.373 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-14T17:13:19.379 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-14T17:13:19.379 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-02-14T17:13:19.385 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2024-02-14T17:13:19.391 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2024-02-14T17:13:19.391 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2024-02-14T17:13:19.391 INFO:teuthology.orchestra.run.smithi082.stdout:Repeating the final host check... 2024-02-14T17:13:19.415 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T17:13:19.415 INFO:teuthology.orchestra.run.smithi082.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-14T17:13:19.416 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl is present 2024-02-14T17:13:19.416 INFO:teuthology.orchestra.run.smithi082.stdout:lvcreate is present 2024-02-14T17:13:19.422 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-14T17:13:19.422 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-14T17:13:19.427 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-14T17:13:19.428 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-02-14T17:13:19.434 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2024-02-14T17:13:19.440 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2024-02-14T17:13:19.440 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2024-02-14T17:13:19.440 INFO:teuthology.orchestra.run.smithi082.stdout:Host looks OK 2024-02-14T17:13:19.440 INFO:teuthology.orchestra.run.smithi082.stdout:Cluster fsid: 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:13:19.440 INFO:teuthology.orchestra.run.smithi082.stdout:Acquiring lock 139751398619744 on /run/cephadm/39c4996e-cb5c-11ee-95b9-87774f69a715.lock 2024-02-14T17:13:19.440 INFO:teuthology.orchestra.run.smithi082.stdout:Lock 139751398619744 acquired on /run/cephadm/39c4996e-cb5c-11ee-95b9-87774f69a715.lock 2024-02-14T17:13:19.440 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 3300 ... 2024-02-14T17:13:19.441 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 6789 ... 2024-02-14T17:13:19.441 INFO:teuthology.orchestra.run.smithi082.stdout:Base mon IP(s) is [172.21.15.82:3300, 172.21.15.82:6789], mon addrv is [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2024-02-14T17:13:19.444 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.82 metric 100 2024-02-14T17:13:19.444 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.82 metric 100 2024-02-14T17:13:19.447 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-14T17:13:19.447 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-14T17:13:19.447 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1782sec hoplimit 64 pref medium 2024-02-14T17:13:19.450 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-14T17:13:19.450 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-14T17:13:19.451 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-14T17:13:19.451 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-14T17:13:19.451 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-02-14T17:13:19.451 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-14T17:13:19.451 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2024-02-14T17:13:19.452 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2024-02-14T17:13:19.452 INFO:teuthology.orchestra.run.smithi082.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-14T17:13:19.452 INFO:teuthology.orchestra.run.smithi082.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-14T17:13:19.453 INFO:teuthology.orchestra.run.smithi082.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:13:19.871 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 2024-02-14T17:13:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:13:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Getting image source signatures 2024-02-14T17:13:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:85deddcd78c19bc18770d0f2ed0eee83c11fab2d5e1d2ebb809a77baeec400f6 2024-02-14T17:13:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-14T17:13:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying config sha256:170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 2024-02-14T17:13:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-14T17:13:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:ceph: stdout ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev) 2024-02-14T17:13:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph version: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev) 2024-02-14T17:13:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:Extracting ceph user uid/gid from container image... 2024-02-14T17:13:21.815 INFO:teuthology.orchestra.run.smithi082.stdout:stat: stdout 167 167 2024-02-14T17:13:21.815 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial keys... 2024-02-14T17:13:22.828 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQCy9MxluArvChAAhbfE8BF+XLy+gyTOOdJH7A== 2024-02-14T17:13:23.743 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQCz9Mxl23lMDRAAurEzVP8TE/8/DoWRBA6KaQ== 2024-02-14T17:13:24.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQC09MxlGBWyAhAAk9DO1pe9mVZij5vnxm2Lsw== 2024-02-14T17:13:24.599 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial monmap... 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool for a [v2:172.21.15.82:3300,v1:172.21.15.82:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:setting min_mon_release = quincy 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: set fsid to 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:13:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mon... 2024-02-14T17:13:26.404 INFO:teuthology.orchestra.run.smithi082.stdout:create mon.a on 2024-02-14T17:13:26.764 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-14T17:13:26.942 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715.target. 2024-02-14T17:13:26.943 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715.target. 2024-02-14T17:13:27.186 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.a 2024-02-14T17:13:27.186 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.a.service: Unit ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.a.service not loaded. 2024-02-14T17:13:27.340 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715.target.wants/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@.service. 2024-02-14T17:13:27.693 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:27 smithi082 podman[25269]: 2024-02-14 17:13:27.575305843 +0000 UTC m=+0.116533668 container create c4f84df53305624b6b471ced00cc26b3e601a891f41d8e5ae6345933828c6c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-14T17:13:27.893 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:13:27.893 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-02-14T17:13:27.899 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:13:27.899 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-02-14T17:13:27.899 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-02-14T17:13:27.899 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2024-02-14T17:13:27.900 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon to start... 2024-02-14T17:13:27.900 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon... 2024-02-14T17:13:28.120 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:27 smithi082 podman[25269]: 2024-02-14 17:13:27.803373059 +0000 UTC m=+0.344600885 container init c4f84df53305624b6b471ced00cc26b3e601a891f41d8e5ae6345933828c6c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-14T17:13:28.120 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:27 smithi082 podman[25269]: 2024-02-14 17:13:27.808611005 +0000 UTC m=+0.349838834 container start c4f84df53305624b6b471ced00cc26b3e601a891f41d8e5ae6345933828c6c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-02-14T17:13:28.121 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:27 smithi082 bash[25269]: c4f84df53305624b6b471ced00cc26b3e601a891f41d8e5ae6345933828c6c31 2024-02-14T17:13:28.121 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:27 smithi082 ceph-mon[25307]: mkfs 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:13:28.121 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:27 smithi082 systemd[1]: Started Ceph mon.a for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:13:28.121 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:27 smithi082 ceph-mon[25307]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-14T17:13:28.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:28 smithi082 ceph-mon[25307]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-14T17:13:28.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:28 smithi082 ceph-mon[25307]: monmap e1: 1 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:13:28.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:28 smithi082 ceph-mon[25307]: fsmap 2024-02-14T17:13:28.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:28 smithi082 ceph-mon[25307]: osdmap e1: 0 total, 0 up, 0 in 2024-02-14T17:13:28.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:28 smithi082 ceph-mon[25307]: mgrmap e1: no daemons active 2024-02-14T17:13:28.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:28 smithi082 ceph-mon[25307]: from='client.? 172.21.15.82:0/3551468540' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-14T17:13:28.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout cluster: 2024-02-14T17:13:28.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout id: 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:13:28.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-14T17:13:28.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:28.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout services: 2024-02-14T17:13:28.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.523984s) 2024-02-14T17:13:28.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-14T17:13:28.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-14T17:13:28.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:28.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout data: 2024-02-14T17:13:28.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-14T17:13:28.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-14T17:13:28.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-14T17:13:28.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pgs: 2024-02-14T17:13:28.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:28.981 INFO:teuthology.orchestra.run.smithi082.stdout:mon is available 2024-02-14T17:13:28.981 INFO:teuthology.orchestra.run.smithi082.stdout:Assimilating anything we can from ceph.conf... 2024-02-14T17:13:30.066 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:29 smithi082 ceph-mon[25307]: from='client.? 172.21.15.82:0/988409818' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-14T17:13:30.066 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:29 smithi082 ceph-mon[25307]: from='client.? 172.21.15.82:0/988409818' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-14T17:13:30.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:30.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2024-02-14T17:13:30.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-14T17:13:30.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-14T17:13:30.071 INFO:teuthology.orchestra.run.smithi082.stdout:Generating new minimal ceph.conf... 2024-02-14T17:13:31.083 INFO:teuthology.orchestra.run.smithi082.stdout:Restarting the monitor... 2024-02-14T17:13:31.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:30 smithi082 ceph-mon[25307]: from='client.? 172.21.15.82:0/1752691318' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:13:31.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:31 smithi082 systemd[1]: Stopping Ceph mon.a for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:13:31.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:31 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a[25284]: 2024-02-14T17:13:31.578+0000 7f1a980d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:13:31.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:31 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a[25284]: 2024-02-14T17:13:31.578+0000 7f1a980d1700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-14T17:13:31.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:31 smithi082 podman[25639]: 2024-02-14 17:13:31.69182182 +0000 UTC m=+0.453235243 container died c4f84df53305624b6b471ced00cc26b3e601a891f41d8e5ae6345933828c6c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-14T17:13:35.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:35 smithi082 podman[25639]: 2024-02-14 17:13:35.11742205 +0000 UTC m=+3.878835479 container cleanup c4f84df53305624b6b471ced00cc26b3e601a891f41d8e5ae6345933828c6c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:13:35.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:35 smithi082 bash[25639]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a 2024-02-14T17:13:37.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:37 smithi082 podman[25658]: 2024-02-14 17:13:37.085745229 +0000 UTC m=+5.385673998 container remove c4f84df53305624b6b471ced00cc26b3e601a891f41d8e5ae6345933828c6c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T17:13:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:37 smithi082 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-14T17:13:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:37 smithi082 systemd[1]: Stopped Ceph mon.a for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:13:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:37 smithi082 systemd[1]: Starting Ceph mon.a for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:13:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:37 smithi082 podman[25759]: 2024-02-14 17:13:37.491300458 +0000 UTC m=+0.017858078 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:13:38.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:38 smithi082 podman[25759]: 2024-02-14 17:13:38.684440499 +0000 UTC m=+1.210998104 container create 9549c93bb8c407dffc00098a9d7a8f6db94d09ae39ce52ea96d8b4860cb5433b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-14T17:13:40.647 INFO:teuthology.orchestra.run.smithi082.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-14T17:13:40.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 podman[25759]: 2024-02-14 17:13:40.414804818 +0000 UTC m=+2.941362423 container init 9549c93bb8c407dffc00098a9d7a8f6db94d09ae39ce52ea96d8b4860cb5433b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-14T17:13:40.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 podman[25759]: 2024-02-14 17:13:40.421462353 +0000 UTC m=+2.948019959 container start 9549c93bb8c407dffc00098a9d7a8f6db94d09ae39ce52ea96d8b4860cb5433b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-14T17:13:40.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T17:13:40.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev), process ceph-mon, pid 7 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: pidfile_write: ignore empty --pid-file 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: load: jerasure load: lrc 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: RocksDB version: 7.9.2 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Git sha 0 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Compile date 2024-02-14 11:59:27 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: DB SUMMARY 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: DB Session ID: ND6P60J1IAYA3ZN3KQTV 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: CURRENT file: CURRENT 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 84619 ; 2024-02-14T17:13:40.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.error_if_exists: 0 2024-02-14T17:13:40.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.create_if_missing: 0 2024-02-14T17:13:40.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.paranoid_checks: 1 2024-02-14T17:13:40.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T17:13:40.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T17:13:40.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T17:13:40.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.env: 0x5640ac411ae0 2024-02-14T17:13:40.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T17:13:40.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.info_log: 0x5640ada4b8c0 2024-02-14T17:13:40.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.statistics: (nil) 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.use_fsync: 0 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_log_file_size: 0 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.allow_fallocate: 1 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.use_direct_reads: 0 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T17:13:40.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.db_log_dir: 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.wal_dir: 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.write_buffer_manager: 0x5640ada1f400 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T17:13:40.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.unordered_write: 0 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.row_cache: None 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.wal_filter: None 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T17:13:40.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.two_write_queues: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.wal_compression: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.atomic_flush: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.log_readahead_size: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T17:13:40.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_background_jobs: 2 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_background_compactions: -1 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_subcompactions: 1 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_open_files: -1 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T17:13:40.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_background_flushes: -1 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Compression algorithms supported: 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: kZSTD supported: 0 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: kXpressCompression supported: 0 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: kZlibCompression supported: 1 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: kSnappyCompression supported: 1 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: kLZ4Compression supported: 1 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: kBZip2Compression supported: 0 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T17:13:40.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.merge_operator: 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_filter: None 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5640ada4b9c0) 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks: 1 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_top_level_index_and_filter: 1 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_type: 0 2024-02-14T17:13:40.742 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_index_type: 0 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_shortening: 1 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: checksum: 4 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: no_block_cache: 0 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache: 0x5640ada4f090 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_name: BinnedLRUCache 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_options: 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: capacity : 536870912 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_shard_bits : 4 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: strict_capacity_limit : 0 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: high_pri_pool_ratio: 0.000 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_compressed: (nil) 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: persistent_cache: (nil) 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size: 4096 2024-02-14T17:13:40.743 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size_deviation: 10 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_restart_interval: 16 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_block_restart_interval: 1 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: metadata_block_size: 4096 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: partition_filters: 0 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: use_delta_encoding: 1 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: filter_policy: bloomfilter 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: whole_key_filtering: 1 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: verify_compression: 0 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: read_amp_bytes_per_bit: 0 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: format_version: 5 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_index_compression: 1 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_align: 0 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: max_auto_readahead_size: 262144 2024-02-14T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi082.stdout: prepopulate_block_cache: 0 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout: initial_auto_readahead_size: 8192 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression: NoCompression 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.num_levels: 7 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T17:13:40.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T17:13:40.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T17:13:40.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T17:13:40.748 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.table_properties_collectors: 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.inplace_update_support: 0 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.bloom_locality: 0 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.max_successive_merges: 0 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T17:13:40.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.ttl: 2592000 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.enable_blob_files: false 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.min_blob_size: 0 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T17:13:40.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 64ae7c9a-9231-4d19-85b5-828bf69db444 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707930820453641, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707930820454576, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81244, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 252, "table_properties": {"data_size": 79431, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 73740, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 211, "num_filter_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707930820, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "64ae7c9a-9231-4d19-85b5-828bf69db444", "db_session_id": "ND6P60J1IAYA3ZN3KQTV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707930820454669, "job": 1, "event": "recovery_finished"} 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5640adb40000 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: DB pointer 0x5640adb2a000 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** DB Stats ** 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:13:40.751 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: L0 2/0 81.17 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 126.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: Sum 2/0 81.17 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 126.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 126.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:13:40.752 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 126.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative compaction: 0.00 GB write, 23.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:13:40.753 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval compaction: 0.00 GB write, 23.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache BinnedLRUCache@0x5640ada4f090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: starting mon.a rank 0 at public addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] at bind addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mon.a@-1(???) e1 preinit fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mon.a@-1(???).mds e1 new map 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mon.a@-1(???).mds e1 print_map 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: e1 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: legacy client fscid: -1 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-02-14T17:13:40.754 INFO:journalctl@ceph.mon.a.smithi082.stdout: No filesystems configured 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: monmap e1: 1 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: fsmap 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: osdmap e1: 0 total, 0 up, 0 in 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 ceph-mon[25796]: mgrmap e1: no daemons active 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 bash[25759]: 9549c93bb8c407dffc00098a9d7a8f6db94d09ae39ce52ea96d8b4860cb5433b 2024-02-14T17:13:40.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:40 smithi082 systemd[1]: Started Ceph mon.a for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:13:41.800 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-14T17:13:41.822 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:13:41.822 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mgr... 2024-02-14T17:13:41.823 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-14T17:13:41.823 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-14T17:13:42.081 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.y 2024-02-14T17:13:42.081 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.y.service: Unit ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.y.service not loaded. 2024-02-14T17:13:42.233 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715.target.wants/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@.service. 2024-02-14T17:13:42.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:42 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/482160272' entity='client.admin' 2024-02-14T17:13:42.596 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:42 smithi082 podman[26041]: 2024-02-14 17:13:42.473339067 +0000 UTC m=+0.119965892 container create ff02decca377922d493ced8757908497b9b9c41fff3e973b00cb7cf31ad517fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-14T17:13:42.822 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:13:42.823 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-02-14T17:13:42.829 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:13:42.830 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-02-14T17:13:42.830 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-02-14T17:13:42.830 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2024-02-14T17:13:42.838 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:13:42.838 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-02-14T17:13:42.845 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:13:42.845 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-02-14T17:13:42.845 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-02-14T17:13:42.845 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-14T17:13:42.845 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr to start... 2024-02-14T17:13:42.845 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr... 2024-02-14T17:13:42.971 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:42 smithi082 podman[26041]: 2024-02-14 17:13:42.70062984 +0000 UTC m=+0.347256665 container init ff02decca377922d493ced8757908497b9b9c41fff3e973b00cb7cf31ad517fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-14T17:13:42.971 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:42 smithi082 podman[26041]: 2024-02-14 17:13:42.705812328 +0000 UTC m=+0.352439154 container start ff02decca377922d493ced8757908497b9b9c41fff3e973b00cb7cf31ad517fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-14T17:13:42.971 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:42 smithi082 bash[26041]: ff02decca377922d493ced8757908497b9b9c41fff3e973b00cb7cf31ad517fb 2024-02-14T17:13:42.971 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:42 smithi082 systemd[1]: Started Ceph mgr.y for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:13:43.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:43 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2098683819' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:13:43.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:43 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:43.279+0000 7fd60c388200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:13:43.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:43 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:43.389+0000 7fd60c388200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:13:44.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:44.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "39c4996e-cb5c-11ee-95b9-87774f69a715", 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:13:44.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:13:44.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:44.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:44.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:13:27.878007+0000", 2024-02-14T17:13:44.056 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:13:44.056 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:44.056 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:13:44.056 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:13:44.056 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (1/15)... 2024-02-14T17:13:44.299 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:44.079+0000 7fd60c388200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:13:44.299 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:44.173+0000 7fd60c388200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:13:44.731 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:44.319+0000 7fd60c388200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:13:44.732 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:44.414+0000 7fd60c388200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:13:44.732 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:44.495+0000 7fd60c388200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:13:45.039 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:44.756+0000 7fd60c388200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:13:45.039 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:44.861+0000 7fd60c388200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:13:45.039 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:44.945+0000 7fd60c388200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:13:45.039 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:45 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:45.037+0000 7fd60c388200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:13:45.481 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:45 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:45.211+0000 7fd60c388200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:13:46.481 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:46 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:46.190+0000 7fd60c388200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:13:46.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:46 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:46.320+0000 7fd60c388200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:13:46.949 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:46 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:46.660+0000 7fd60c388200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:13:46.949 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:46 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:46.750+0000 7fd60c388200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:13:46.949 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:46 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:46.947+0000 7fd60c388200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:13:47.216 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:47 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:47.136+0000 7fd60c388200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:13:47.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:47 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/360608369' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:13:47.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:47 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:47.427+0000 7fd60c388200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:13:47.813 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:47 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:47.689+0000 7fd60c388200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:13:47.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:47.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:13:47.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "39c4996e-cb5c-11ee-95b9-87774f69a715", 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:47.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:13:47.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:13:47.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:13:47.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:13:27.878007+0000", 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:13:47.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:47.827 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:13:47.827 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:13:47.827 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (2/15)... 2024-02-14T17:13:48.231 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:47 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:47.925+0000 7fd60c388200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:13:48.981 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:48 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:48.722+0000 7fd60c388200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:13:49.599 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:49.287+0000 7fd60c388200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:13:49.599 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:49.384+0000 7fd60c388200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:13:49.599 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:49.597+0000 7fd60c388200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:13:49.942 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: Activating manager daemon y 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: mgrmap e2: y(active, starting, since 0.00302112s) 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: Manager daemon y is now available 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' 2024-02-14T17:13:49.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' 2024-02-14T17:13:49.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:49 smithi082 ceph-mon[25796]: from='mgr.14100 172.21.15.82:0/3626822462' entity='mgr.y' 2024-02-14T17:13:50.952 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:50 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4073772382' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:13:50.952 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:50 smithi082 ceph-mon[25796]: mgrmap e3: y(active, since 1.00619s) 2024-02-14T17:13:50.957 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:50.957 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:13:50.957 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "39c4996e-cb5c-11ee-95b9-87774f69a715", 2024-02-14T17:13:50.957 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:13:50.957 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:13:50.957 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:13:50.957 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:13:50.957 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:50.957 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:50.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:13:50.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:13:50.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:13:50.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:13:50.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:13:50.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:13:50.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:50.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:13:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:13:27.878007+0000", 2024-02-14T17:13:50.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:13:50.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:50.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:13:50.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:13:50.965 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (3/15)... 2024-02-14T17:13:52.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:52 smithi082 ceph-mon[25796]: mgrmap e4: y(active, since 2s) 2024-02-14T17:13:53.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:53 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2108034370' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "39c4996e-cb5c-11ee-95b9-87774f69a715", 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:13:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:13:54.409 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:13:54.410 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:13:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:13:27.878007+0000", 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:13:54.412 INFO:teuthology.orchestra.run.smithi082.stdout:mgr is available 2024-02-14T17:13:55.356 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:55 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2494114037' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-14T17:13:55.356 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:55 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2494114037' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-14T17:13:55.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:55.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2024-02-14T17:13:55.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:13:55.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-14T17:13:55.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-14T17:13:55.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-14T17:13:55.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-14T17:13:55.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:55.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-14T17:13:55.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-14T17:13:55.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-02-14T17:13:55.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2024-02-14T17:13:55.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-14T17:13:55.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-14T17:13:55.627 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling cephadm module... 2024-02-14T17:13:56.636 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:56 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/3241507554' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-14T17:13:56.636 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:56 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ignoring --setuser ceph since I am not root 2024-02-14T17:13:56.636 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:56 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ignoring --setgroup ceph since I am not root 2024-02-14T17:13:57.192 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:56 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:56.931+0000 7f98ca986200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:13:57.192 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:57 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:57.029+0000 7f98ca986200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:13:57.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:57 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/3241507554' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-14T17:13:57.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:57 smithi082 ceph-mon[25796]: mgrmap e5: y(active, since 6s) 2024-02-14T17:13:57.981 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:57 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:57.685+0000 7f98ca986200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:13:57.982 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:57 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:57.782+0000 7f98ca986200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:13:57.982 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:57 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:57.923+0000 7f98ca986200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:13:58.309 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:58 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:58.021+0000 7f98ca986200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:13:58.309 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:58 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:58.104+0000 7f98ca986200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:13:58.312 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:13:58.313 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-14T17:13:58.313 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T17:13:58.313 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-14T17:13:58.313 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-14T17:13:58.313 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:13:58.313 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2024-02-14T17:13:58.313 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 5... 2024-02-14T17:13:58.572 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:13:58 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/3312868022' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-14T17:13:58.572 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:58 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:58.375+0000 7f98ca986200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:13:58.572 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:58 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:58.485+0000 7f98ca986200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:13:58.573 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:58 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:58.570+0000 7f98ca986200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:13:58.838 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:58 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:58.670+0000 7f98ca986200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:13:59.231 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:58 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:58.848+0000 7f98ca986200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:14:00.231 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:59 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:59.798+0000 7f98ca986200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:14:00.231 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:13:59 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:13:59.928+0000 7f98ca986200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:14:00.540 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:00 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:00.256+0000 7f98ca986200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:14:00.540 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:00 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:00.345+0000 7f98ca986200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:14:00.540 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:00 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:00.538+0000 7f98ca986200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:14:00.981 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:00 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:00.721+0000 7f98ca986200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:14:01.264 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:01 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:01.006+0000 7f98ca986200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:14:01.265 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:01 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:01.262+0000 7f98ca986200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:14:01.731 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:01 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:01.496+0000 7f98ca986200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:14:02.731 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:02 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:02.290+0000 7f98ca986200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:14:03.166 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:02 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:02.852+0000 7f98ca986200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:14:03.166 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:02 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:02.949+0000 7f98ca986200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:14:03.166 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:03.164+0000 7f98ca986200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:14:03.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: Active manager daemon y restarted 2024-02-14T17:14:03.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: Activating manager daemon y 2024-02-14T17:14:03.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: osdmap e2: 0 total, 0 up, 0 in 2024-02-14T17:14:03.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: mgrmap e6: y(active, starting, since 0.00359792s) 2024-02-14T17:14:03.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:14:03.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:14:03.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:14:03.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:14:03.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:14:03.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: Manager daemon y is now available 2024-02-14T17:14:03.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:03.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:03.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:03.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:03 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:04.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:04 smithi082 ceph-mon[25796]: Found migration_current of "None". Setting to last migration. 2024-02-14T17:14:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:04 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:04 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:14:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:04 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:14:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:04 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:04 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:04 smithi082 ceph-mon[25796]: mgrmap e7: y(active, since 1.00591s) 2024-02-14T17:14:04.743 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:14:04.743 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-14T17:14:04.743 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-14T17:14:04.743 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:14:04.743 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 5 is available 2024-02-14T17:14:04.743 INFO:teuthology.orchestra.run.smithi082.stdout:Setting orchestrator backend to cephadm... 2024-02-14T17:14:05.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:05 smithi082 ceph-mon[25796]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-14T17:14:05.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:05 smithi082 ceph-mon[25796]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-14T17:14:05.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:05 smithi082 ceph-mon[25796]: [14/Feb/2024:17:14:04] ENGINE Bus STARTING 2024-02-14T17:14:05.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:05 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:06.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:06 smithi082 ceph-mon[25796]: [14/Feb/2024:17:14:04] ENGINE Serving on https://172.21.15.82:7150 2024-02-14T17:14:06.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:06 smithi082 ceph-mon[25796]: [14/Feb/2024:17:14:04] ENGINE Serving on http://172.21.15.82:8765 2024-02-14T17:14:06.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:06 smithi082 ceph-mon[25796]: [14/Feb/2024:17:14:04] ENGINE Bus STARTED 2024-02-14T17:14:06.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:06 smithi082 ceph-mon[25796]: mgrmap e8: y(active, since 2s) 2024-02-14T17:14:06.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:06 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:06.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:06 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:07.204 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-14T17:14:07.204 INFO:teuthology.orchestra.run.smithi082.stdout:Generating ssh key... 2024-02-14T17:14:07.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-mon[25796]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:08.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:08 smithi082 ceph-mon[25796]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:08.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:08 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:08.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:08 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:08.324 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: Generating public/private rsa key pair. 2024-02-14T17:14:08.324 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: Your identification has been saved in /tmp/tmpd4h1hwyq/key. 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: Your public key has been saved in /tmp/tmpd4h1hwyq/key.pub. 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: The key fingerprint is: 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: SHA256:eIPcuhZZGeizv6fm02A4cAAhKpnWwD3iPqY+jp/7kvU ceph-39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: The key's randomart image is: 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: +---[RSA 3072]----+ 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: |o.+o . | 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: |.*oo. . . | 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: |*....o o | 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: |o. ..++o | 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: |. o+*S | 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: | + . *oo. | 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: |o .o ..= o | 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: |o.o. E.+ o | 2024-02-14T17:14:08.325 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: |++=+...o+= | 2024-02-14T17:14:08.326 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:07 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: +----[SHA256]-----+ 2024-02-14T17:14:09.309 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:09 smithi082 ceph-mon[25796]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:09.309 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:09 smithi082 ceph-mon[25796]: Generating ssh key... 2024-02-14T17:14:09.892 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCovXViOyS8nVdB4C5q5TaGCbAYnmdXfR7p2q4UgOPclJ0uOysbyNSFsjkaQ7kLS6iVY1iuMRyEOB7u2QJcL4NPjQZ3eoFQaaSxPH+vIeTzHf6QRhjVhLwxR9wAoXrutcwjhfTNZXhXy284ywqXsDL+sdYy27zGY7KN8g97/g5Xw0Zu9DhwNNUHPaa5Z+PP/sJ8Lqloddox6lLb8ADAl9oEuwd1xOvzLNO7GPtrJ2+daYhUA2BTCQL6z+g8mTuHE3/wiPMSR0nE4Pecd1MujrGUziTC7At9bTKgMZZvGN0dcvcp5LMsCE57Uxa1xaV3amOH/TEbbBMcdudn+RdqH8UjIWLnDLMzr/YwcFrl9pxiMdFun4r0lPEP5oN/wueCXXR7ktcjbxbPS3yx9JrKQH0o6nwjzy3R8ODGcTYfiARHusHGdD8xc80uURflGAFXxVh10ygkuRNl7KhfBRj7stlkoT8BrFpMcE5UpMWxRkan9JBnujWZh8LiIBpEtkn80bc= ceph-39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:14:09.892 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-14T17:14:09.892 INFO:teuthology.orchestra.run.smithi082.stdout:Adding key to root@localhost authorized_keys... 2024-02-14T17:14:09.892 INFO:teuthology.orchestra.run.smithi082.stdout:Adding host smithi082... 2024-02-14T17:14:10.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:10 smithi082 ceph-mon[25796]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:11.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:11 smithi082 ceph-mon[25796]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:11.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:11 smithi082 ceph-mon[25796]: Deploying cephadm binary to smithi082 2024-02-14T17:14:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Added host 'smithi082' with addr '172.21.15.82' 2024-02-14T17:14:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mon service... 2024-02-14T17:14:16.188 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:15 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:16.191 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:15 smithi082 ceph-mon[25796]: Added host smithi082 2024-02-14T17:14:16.192 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:15 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:16.941 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-14T17:14:16.941 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mgr service... 2024-02-14T17:14:17.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:17 smithi082 ceph-mon[25796]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:17.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:17 smithi082 ceph-mon[25796]: Saving service mon spec with placement count:5 2024-02-14T17:14:17.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:17 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:18.803 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-14T17:14:18.926 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:18 smithi082 ceph-mon[25796]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:18.927 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:18 smithi082 ceph-mon[25796]: Saving service mgr spec with placement count:2 2024-02-14T17:14:18.927 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:18 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:18.927 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:18 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:20.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:19 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:20.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:19 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/3488586137' entity='client.admin' 2024-02-14T17:14:22.019 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling the dashboard module... 2024-02-14T17:14:22.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:21 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:22.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:21 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/299856734' entity='client.admin' 2024-02-14T17:14:23.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:23 smithi082 ceph-mon[25796]: from='mgr.14120 172.21.15.82:0/659280487' entity='mgr.y' 2024-02-14T17:14:23.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:23 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1547130931' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-14T17:14:23.475 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:23 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ignoring --setuser ceph since I am not root 2024-02-14T17:14:23.475 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:23 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ignoring --setgroup ceph since I am not root 2024-02-14T17:14:23.731 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:23 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:23.646+0000 7faed78df200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:14:24.062 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:23 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:23.741+0000 7faed78df200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:14:24.354 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:24 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1547130931' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-14T17:14:24.354 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:24 smithi082 ceph-mon[25796]: mgrmap e9: y(active, since 19s) 2024-02-14T17:14:24.632 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:24 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:24.395+0000 7faed78df200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:14:24.632 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:24 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:24.488+0000 7faed78df200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:14:24.632 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:24 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:24.631+0000 7faed78df200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:14:24.940 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:24 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:24.729+0000 7faed78df200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:14:24.941 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:24 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:24.815+0000 7faed78df200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:14:25.199 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:25 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:25.086+0000 7faed78df200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:14:25.199 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:25 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:25.198+0000 7faed78df200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:14:25.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:25 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/3504184882' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-14T17:14:25.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:25 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:25.289+0000 7faed78df200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:14:25.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:25 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:25.384+0000 7faed78df200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:14:25.835 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:25 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:25.562+0000 7faed78df200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:14:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:14:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-14T17:14:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T17:14:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-14T17:14:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-14T17:14:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:14:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2024-02-14T17:14:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 9... 2024-02-14T17:14:26.982 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:26 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:26.534+0000 7faed78df200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:14:26.982 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:26 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:26.663+0000 7faed78df200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:14:27.280 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:26 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:26.992+0000 7faed78df200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:14:27.281 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:27 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:27.084+0000 7faed78df200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:14:27.281 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:27 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:27.279+0000 7faed78df200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:14:27.731 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:27 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:27.462+0000 7faed78df200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:14:28.012 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:27 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:27.747+0000 7faed78df200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:14:28.012 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:28 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:28.011+0000 7faed78df200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:14:28.481 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:28 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:28.246+0000 7faed78df200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:14:29.481 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:29.044+0000 7faed78df200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:14:29.919 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:29.604+0000 7faed78df200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:14:29.919 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:29.703+0000 7faed78df200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:14:29.920 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:14:29.918+0000 7faed78df200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:14:30.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: Active manager daemon y restarted 2024-02-14T17:14:30.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: Activating manager daemon y 2024-02-14T17:14:30.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: osdmap e3: 0 total, 0 up, 0 in 2024-02-14T17:14:30.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: mgrmap e10: y(active, starting, since 0.0030341s) 2024-02-14T17:14:30.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:14:30.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:14:30.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:14:30.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:14:30.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:14:30.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: Manager daemon y is now available 2024-02-14T17:14:30.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:30.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:31.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:30 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:14:31.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:30 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:14:31.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:30 smithi082 ceph-mon[25796]: mgrmap e11: y(active, since 1.00573s) 2024-02-14T17:14:31.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:14:31.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-14T17:14:31.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-14T17:14:31.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:14:31.513 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 9 is available 2024-02-14T17:14:31.513 INFO:teuthology.orchestra.run.smithi082.stdout:Generating a dashboard self-signed certificate... 2024-02-14T17:14:32.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:31 smithi082 ceph-mon[25796]: [14/Feb/2024:17:14:30] ENGINE Bus STARTING 2024-02-14T17:14:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:31 smithi082 ceph-mon[25796]: [14/Feb/2024:17:14:30] ENGINE Serving on https://172.21.15.82:7150 2024-02-14T17:14:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:31 smithi082 ceph-mon[25796]: [14/Feb/2024:17:14:30] ENGINE Serving on http://172.21.15.82:8765 2024-02-14T17:14:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:31 smithi082 ceph-mon[25796]: [14/Feb/2024:17:14:30] ENGINE Bus STARTED 2024-02-14T17:14:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:31 smithi082 ceph-mon[25796]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-14T17:14:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:31 smithi082 ceph-mon[25796]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-14T17:14:32.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-14T17:14:32.937 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial admin user... 2024-02-14T17:14:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:32 smithi082 ceph-mon[25796]: mgrmap e12: y(active, since 2s) 2024-02-14T17:14:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:34.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:33 smithi082 ceph-mon[25796]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:34.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:33 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:34.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:33 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:34.526 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$KETYI3Ek/QpsQK/bkuW8UubKBxhRihaOJA891b2jCoD5JUPDGsJ/y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707930873, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-14T17:14:34.526 INFO:teuthology.orchestra.run.smithi082.stdout:Fetching dashboard port number... 2024-02-14T17:14:35.186 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:34 smithi082 ceph-mon[25796]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:36.326 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:35 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/3182617907' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-14T17:14:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 8443 2024-02-14T17:14:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:14:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-02-14T17:14:36.348 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:14:36.348 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-02-14T17:14:36.348 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-02-14T17:14:36.348 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-14T17:14:36.348 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph Dashboard is now available at: 2024-02-14T17:14:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: URL: https://smithi082.front.sepia.ceph.com:8443/ 2024-02-14T17:14:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: User: admin 2024-02-14T17:14:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: Password: j87vrpoug5 2024-02-14T17:14:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:36.349 INFO:teuthology.orchestra.run.smithi082.stdout:Saving cluster configuration to /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config directory 2024-02-14T17:14:36.349 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling autotune for osd_memory_target 2024-02-14T17:14:37.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:36 smithi082 ceph-mon[25796]: mgrmap e13: y(active, since 6s) 2024-02-14T17:14:39.301 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-14T17:14:39.301 INFO:teuthology.orchestra.run.smithi082.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-14T17:14:39.301 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:39.301 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:14:39.301 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:39.301 INFO:teuthology.orchestra.run.smithi082.stdout:Or, if you are only running a single cluster on this host: 2024-02-14T17:14:39.301 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:39.301 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout: ceph telemetry on 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout:For more information see: 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:39.302 INFO:teuthology.orchestra.run.smithi082.stdout:Bootstrap complete. 2024-02-14T17:14:39.341 INFO:tasks.cephadm:Fetching config... 2024-02-14T17:14:39.341 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:14:39.341 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-14T17:14:39.356 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-14T17:14:39.357 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:14:39.357 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-14T17:14:39.412 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-14T17:14:39.412 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:14:39.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-14T17:14:39.479 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-14T17:14:39.479 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:14:39.479 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-14T17:14:39.534 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-14T17:14:39.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCovXViOyS8nVdB4C5q5TaGCbAYnmdXfR7p2q4UgOPclJ0uOysbyNSFsjkaQ7kLS6iVY1iuMRyEOB7u2QJcL4NPjQZ3eoFQaaSxPH+vIeTzHf6QRhjVhLwxR9wAoXrutcwjhfTNZXhXy284ywqXsDL+sdYy27zGY7KN8g97/g5Xw0Zu9DhwNNUHPaa5Z+PP/sJ8Lqloddox6lLb8ADAl9oEuwd1xOvzLNO7GPtrJ2+daYhUA2BTCQL6z+g8mTuHE3/wiPMSR0nE4Pecd1MujrGUziTC7At9bTKgMZZvGN0dcvcp5LMsCE57Uxa1xaV3amOH/TEbbBMcdudn+RdqH8UjIWLnDLMzr/YwcFrl9pxiMdFun4r0lPEP5oN/wueCXXR7ktcjbxbPS3yx9JrKQH0o6nwjzy3R8ODGcTYfiARHusHGdD8xc80uURflGAFXxVh10ygkuRNl7KhfBRj7stlkoT8BrFpMcE5UpMWxRkan9JBnujWZh8LiIBpEtkn80bc= ceph-39c4996e-cb5c-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T17:14:39.612 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCovXViOyS8nVdB4C5q5TaGCbAYnmdXfR7p2q4UgOPclJ0uOysbyNSFsjkaQ7kLS6iVY1iuMRyEOB7u2QJcL4NPjQZ3eoFQaaSxPH+vIeTzHf6QRhjVhLwxR9wAoXrutcwjhfTNZXhXy284ywqXsDL+sdYy27zGY7KN8g97/g5Xw0Zu9DhwNNUHPaa5Z+PP/sJ8Lqloddox6lLb8ADAl9oEuwd1xOvzLNO7GPtrJ2+daYhUA2BTCQL6z+g8mTuHE3/wiPMSR0nE4Pecd1MujrGUziTC7At9bTKgMZZvGN0dcvcp5LMsCE57Uxa1xaV3amOH/TEbbBMcdudn+RdqH8UjIWLnDLMzr/YwcFrl9pxiMdFun4r0lPEP5oN/wueCXXR7ktcjbxbPS3yx9JrKQH0o6nwjzy3R8ODGcTYfiARHusHGdD8xc80uURflGAFXxVh10ygkuRNl7KhfBRj7stlkoT8BrFpMcE5UpMWxRkan9JBnujWZh8LiIBpEtkn80bc= ceph-39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:14:39.627 DEBUG:teuthology.orchestra.run.smithi091:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCovXViOyS8nVdB4C5q5TaGCbAYnmdXfR7p2q4UgOPclJ0uOysbyNSFsjkaQ7kLS6iVY1iuMRyEOB7u2QJcL4NPjQZ3eoFQaaSxPH+vIeTzHf6QRhjVhLwxR9wAoXrutcwjhfTNZXhXy284ywqXsDL+sdYy27zGY7KN8g97/g5Xw0Zu9DhwNNUHPaa5Z+PP/sJ8Lqloddox6lLb8ADAl9oEuwd1xOvzLNO7GPtrJ2+daYhUA2BTCQL6z+g8mTuHE3/wiPMSR0nE4Pecd1MujrGUziTC7At9bTKgMZZvGN0dcvcp5LMsCE57Uxa1xaV3amOH/TEbbBMcdudn+RdqH8UjIWLnDLMzr/YwcFrl9pxiMdFun4r0lPEP5oN/wueCXXR7ktcjbxbPS3yx9JrKQH0o6nwjzy3R8ODGcTYfiARHusHGdD8xc80uURflGAFXxVh10ygkuRNl7KhfBRj7stlkoT8BrFpMcE5UpMWxRkan9JBnujWZh8LiIBpEtkn80bc= ceph-39c4996e-cb5c-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T17:14:39.675 INFO:teuthology.orchestra.run.smithi091.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCovXViOyS8nVdB4C5q5TaGCbAYnmdXfR7p2q4UgOPclJ0uOysbyNSFsjkaQ7kLS6iVY1iuMRyEOB7u2QJcL4NPjQZ3eoFQaaSxPH+vIeTzHf6QRhjVhLwxR9wAoXrutcwjhfTNZXhXy284ywqXsDL+sdYy27zGY7KN8g97/g5Xw0Zu9DhwNNUHPaa5Z+PP/sJ8Lqloddox6lLb8ADAl9oEuwd1xOvzLNO7GPtrJ2+daYhUA2BTCQL6z+g8mTuHE3/wiPMSR0nE4Pecd1MujrGUziTC7At9bTKgMZZvGN0dcvcp5LMsCE57Uxa1xaV3amOH/TEbbBMcdudn+RdqH8UjIWLnDLMzr/YwcFrl9pxiMdFun4r0lPEP5oN/wueCXXR7ktcjbxbPS3yx9JrKQH0o6nwjzy3R8ODGcTYfiARHusHGdD8xc80uURflGAFXxVh10ygkuRNl7KhfBRj7stlkoT8BrFpMcE5UpMWxRkan9JBnujWZh8LiIBpEtkn80bc= ceph-39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:14:39.690 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCovXViOyS8nVdB4C5q5TaGCbAYnmdXfR7p2q4UgOPclJ0uOysbyNSFsjkaQ7kLS6iVY1iuMRyEOB7u2QJcL4NPjQZ3eoFQaaSxPH+vIeTzHf6QRhjVhLwxR9wAoXrutcwjhfTNZXhXy284ywqXsDL+sdYy27zGY7KN8g97/g5Xw0Zu9DhwNNUHPaa5Z+PP/sJ8Lqloddox6lLb8ADAl9oEuwd1xOvzLNO7GPtrJ2+daYhUA2BTCQL6z+g8mTuHE3/wiPMSR0nE4Pecd1MujrGUziTC7At9bTKgMZZvGN0dcvcp5LMsCE57Uxa1xaV3amOH/TEbbBMcdudn+RdqH8UjIWLnDLMzr/YwcFrl9pxiMdFun4r0lPEP5oN/wueCXXR7ktcjbxbPS3yx9JrKQH0o6nwjzy3R8ODGcTYfiARHusHGdD8xc80uURflGAFXxVh10ygkuRNl7KhfBRj7stlkoT8BrFpMcE5UpMWxRkan9JBnujWZh8LiIBpEtkn80bc= ceph-39c4996e-cb5c-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T17:14:39.736 INFO:teuthology.orchestra.run.smithi120.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCovXViOyS8nVdB4C5q5TaGCbAYnmdXfR7p2q4UgOPclJ0uOysbyNSFsjkaQ7kLS6iVY1iuMRyEOB7u2QJcL4NPjQZ3eoFQaaSxPH+vIeTzHf6QRhjVhLwxR9wAoXrutcwjhfTNZXhXy284ywqXsDL+sdYy27zGY7KN8g97/g5Xw0Zu9DhwNNUHPaa5Z+PP/sJ8Lqloddox6lLb8ADAl9oEuwd1xOvzLNO7GPtrJ2+daYhUA2BTCQL6z+g8mTuHE3/wiPMSR0nE4Pecd1MujrGUziTC7At9bTKgMZZvGN0dcvcp5LMsCE57Uxa1xaV3amOH/TEbbBMcdudn+RdqH8UjIWLnDLMzr/YwcFrl9pxiMdFun4r0lPEP5oN/wueCXXR7ktcjbxbPS3yx9JrKQH0o6nwjzy3R8ODGcTYfiARHusHGdD8xc80uURflGAFXxVh10ygkuRNl7KhfBRj7stlkoT8BrFpMcE5UpMWxRkan9JBnujWZh8LiIBpEtkn80bc= ceph-39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:14:39.750 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-14T17:14:39.852 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:39 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/3587190311' entity='client.admin' 2024-02-14T17:14:39.994 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:14:43.299 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-14T17:14:43.299 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-14T17:14:43.951 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:43 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1995789141' entity='client.admin' 2024-02-14T17:14:44.066 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:14:45.124 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:45.124 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:45.124 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:14:45.124 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:45.124 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:45.125 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:46.586 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi091 2024-02-14T17:14:46.587 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:14:46.587 DEBUG:teuthology.orchestra.run.smithi091:> dd of=/etc/ceph/ceph.conf 2024-02-14T17:14:46.605 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:14:46.605 DEBUG:teuthology.orchestra.run.smithi091:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:14:46.662 INFO:tasks.cephadm:Adding host smithi091 to orchestrator... 2024-02-14T17:14:46.662 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch host add smithi091 2024-02-14T17:14:46.898 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:14:47.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:47.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:47.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:47.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:14:47.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:14:47.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: Updating smithi082:/etc/ceph/ceph.conf 2024-02-14T17:14:47.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:14:47.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:47.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:47.120 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:46 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:47.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:47 smithi082 ceph-mon[25796]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:14:47.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:47 smithi082 ceph-mon[25796]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:14:49.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:49 smithi082 ceph-mon[25796]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi091", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:14:51.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:50 smithi082 ceph-mon[25796]: Deploying cephadm binary to smithi091 2024-02-14T17:14:51.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:50 smithi082 ceph-mon[25796]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:14:53.036 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi091' with addr '172.21.15.91' 2024-02-14T17:14:53.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:52 smithi082 ceph-mon[25796]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:14:53.587 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-14T17:14:53.818 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:14:54.199 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:54 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:54.199 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:54 smithi082 ceph-mon[25796]: Added host smithi091 2024-02-14T17:14:54.199 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:54 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:14:54.199 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:54 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:55.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:55 smithi082 ceph-mon[25796]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:14:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:55 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:55.716 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:14:55.716 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.91", "hostname": "smithi091", "labels": [], "status": ""}] 2024-02-14T17:14:56.362 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi120 2024-02-14T17:14:56.362 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:14:56.362 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.conf 2024-02-14T17:14:56.383 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:14:56.383 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:14:56.440 INFO:tasks.cephadm:Adding host smithi120 to orchestrator... 2024-02-14T17:14:56.440 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch host add smithi120 2024-02-14T17:14:56.675 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:14:56.886 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:56 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:14:56.887 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:56 smithi082 ceph-mon[25796]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:14:56.887 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:56 smithi082 ceph-mon[25796]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:14:59.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:14:58 smithi082 ceph-mon[25796]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:00.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:00 smithi082 ceph-mon[25796]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi091", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:15:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:00 smithi082 ceph-mon[25796]: Updating smithi091:/etc/ceph/ceph.conf 2024-02-14T17:15:01.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:01 smithi082 ceph-mon[25796]: Deploying cephadm binary to smithi120 2024-02-14T17:15:01.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:01 smithi082 ceph-mon[25796]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:01.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:01 smithi082 ceph-mon[25796]: Updating smithi091:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:15:01.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:01 smithi082 ceph-mon[25796]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:15:01.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:01 smithi082 ceph-mon[25796]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:02.800 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi120' with addr '172.21.15.120' 2024-02-14T17:15:03.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:03 smithi082 ceph-mon[25796]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:03.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:03 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:03.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:03 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:03.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:03 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:03.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:03 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:03.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:03 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:03.441 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-14T17:15:03.674 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:15:04.081 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:04 smithi082 ceph-mon[25796]: Added host smithi120 2024-02-14T17:15:04.081 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:04 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:04.081 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:04 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:05.311 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:05 smithi082 ceph-mon[25796]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:05.531 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:15:05.531 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.91", "hostname": "smithi091", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}] 2024-02-14T17:15:06.303 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-14T17:15:06.303 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd crush tunables default 2024-02-14T17:15:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:15:06.658 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:06 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:06.658 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:06 smithi082 ceph-mon[25796]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:15:06.658 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:06 smithi082 ceph-mon[25796]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2024-02-14T17:15:09.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:08 smithi082 ceph-mon[25796]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:09.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:08 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/798761070' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T17:15:09.728 INFO:tasks.cephadm:Adding mon.a on smithi082 2024-02-14T17:15:09.728 INFO:tasks.cephadm:Adding mon.b on smithi091 2024-02-14T17:15:09.728 INFO:tasks.cephadm:Adding mon.c on smithi120 2024-02-14T17:15:09.729 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch apply mon '3;smithi082:172.21.15.82=a;smithi091:172.21.15.91=b;smithi120:172.21.15.120=c' 2024-02-14T17:15:09.958 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/798761070' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:10.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: Updating smithi120:/etc/ceph/ceph.conf 2024-02-14T17:15:10.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:10.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:10.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:11.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:10 smithi082 ceph-mon[25796]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:11.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:10 smithi082 ceph-mon[25796]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:15:11.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:10 smithi082 ceph-mon[25796]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:15:11.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:10 smithi082 ceph-mon[25796]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:11.725 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled mon update... 2024-02-14T17:15:12.348 DEBUG:teuthology.orchestra.run.smithi091:mon.b> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.b.service 2024-02-14T17:15:12.351 DEBUG:teuthology.orchestra.run.smithi120:mon.c> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.c.service 2024-02-14T17:15:12.353 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:15:12.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:15:12.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi091:172.21.15.91=b;smithi120:172.21.15.120=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:12.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi091:172.21.15.91=b;smithi120:172.21.15.120=c;count:3 2024-02-14T17:15:12.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:12.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:12.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:12.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:12.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:12.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:12.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:12.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: Deploying daemon mon.c on smithi120 2024-02-14T17:15:12.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:12 smithi082 ceph-mon[25796]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:15:15.100 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":1,"fsid":"39c4996e-cb5c-11ee-95b9-87774f69a715","modified":"2024-02-14T17:13:24.964428Z","created":"2024-02-14T17:13:24.964428Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-14T17:15:15.102 INFO:teuthology.orchestra.run.smithi120.stderr:dumped monmap epoch 1 2024-02-14T17:15:15.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:14 smithi082 ceph-mon[25796]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:16.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:15 smithi082 ceph-mon[25796]: from='client.? 172.21.15.120:0/159723773' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:15:17.226 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 podman[30861]: 2024-02-14 17:15:17.045521668 +0000 UTC m=+0.017472307 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:15:17.312 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:15:17.312 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:15:17.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:16 smithi082 ceph-mon[25796]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 podman[30861]: 2024-02-14 17:15:17.30235842 +0000 UTC m=+0.274309050 container init 40bbe8467118cf800f0b81455edfb8d0a2e1fe7549c27195ca3ff39ef1e139c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 podman[30861]: 2024-02-14 17:15:17.308249822 +0000 UTC m=+0.280200455 container start 40bbe8467118cf800f0b81455edfb8d0a2e1fe7549c27195ca3ff39ef1e139c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-c, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240214) 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev), process ceph-mon, pid 7 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pidfile_write: ignore empty --pid-file 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: load: jerasure load: lrc 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: RocksDB version: 7.9.2 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Git sha 0 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Compile date 2024-02-14 11:59:27 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: DB SUMMARY 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: DB Session ID: 431WNKQ1510QUB9DM0NU 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: CURRENT file: CURRENT 2024-02-14T17:15:17.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.error_if_exists: 0 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.create_if_missing: 0 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.paranoid_checks: 1 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.env: 0x5579c19d3ae0 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.info_log: 0x5579c325b8c0 2024-02-14T17:15:17.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T17:15:17.559 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.statistics: (nil) 2024-02-14T17:15:17.559 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.use_fsync: 0 2024-02-14T17:15:17.559 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_log_file_size: 0 2024-02-14T17:15:17.559 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T17:15:17.559 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T17:15:17.559 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.allow_fallocate: 1 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.use_direct_reads: 0 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.db_log_dir: 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.wal_dir: 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T17:15:17.560 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.write_buffer_manager: 0x5579c322f400 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T17:15:17.561 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.unordered_write: 0 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.row_cache: None 2024-02-14T17:15:17.562 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.wal_filter: None 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.two_write_queues: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.wal_compression: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.atomic_flush: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.log_readahead_size: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T17:15:17.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T17:15:17.564 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T17:15:17.564 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T17:15:17.564 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T17:15:17.564 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T17:15:17.564 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_background_jobs: 2 2024-02-14T17:15:17.564 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_background_compactions: -1 2024-02-14T17:15:17.564 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_subcompactions: 1 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_open_files: -1 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T17:15:17.565 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_background_flushes: -1 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Compression algorithms supported: 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: kZSTD supported: 0 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: kXpressCompression supported: 0 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: kZlibCompression supported: 1 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: kSnappyCompression supported: 1 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: kLZ4Compression supported: 1 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: kBZip2Compression supported: 0 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T17:15:17.566 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.merge_operator: 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_filter: None 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5579c325b9c0) 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout: cache_index_and_filter_blocks: 1 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout: pin_top_level_index_and_filter: 1 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout: index_type: 0 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout: data_block_index_type: 0 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout: index_shortening: 1 2024-02-14T17:15:17.567 INFO:journalctl@ceph.mon.c.smithi120.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: checksum: 4 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: no_block_cache: 0 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: block_cache: 0x5579c325f090 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: block_cache_name: BinnedLRUCache 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: block_cache_options: 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: capacity : 536870912 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: num_shard_bits : 4 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: strict_capacity_limit : 0 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: high_pri_pool_ratio: 0.000 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: block_cache_compressed: (nil) 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: persistent_cache: (nil) 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: block_size: 4096 2024-02-14T17:15:17.568 INFO:journalctl@ceph.mon.c.smithi120.stdout: block_size_deviation: 10 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: block_restart_interval: 16 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: index_block_restart_interval: 1 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: metadata_block_size: 4096 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: partition_filters: 0 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: use_delta_encoding: 1 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: filter_policy: bloomfilter 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: whole_key_filtering: 1 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: verify_compression: 0 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: read_amp_bytes_per_bit: 0 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: format_version: 5 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: enable_index_compression: 1 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: block_align: 0 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: max_auto_readahead_size: 262144 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: prepopulate_block_cache: 0 2024-02-14T17:15:17.569 INFO:journalctl@ceph.mon.c.smithi120.stdout: initial_auto_readahead_size: 8192 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression: NoCompression 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.num_levels: 7 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T17:15:17.570 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:15:17.571 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T17:15:17.572 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T17:15:17.572 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T17:15:17.572 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T17:15:17.572 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T17:15:17.572 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T17:15:17.572 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T17:15:17.572 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T17:15:17.572 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T17:15:17.572 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T17:15:17.573 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T17:15:17.574 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T17:15:17.574 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T17:15:17.574 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T17:15:17.574 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T17:15:17.574 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T17:15:17.574 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T17:15:17.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T17:15:17.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T17:15:17.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T17:15:17.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T17:15:17.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T17:15:17.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T17:15:17.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.table_properties_collectors: 2024-02-14T17:15:17.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.inplace_update_support: 0 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.bloom_locality: 0 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.max_successive_merges: 0 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.ttl: 2592000 2024-02-14T17:15:17.576 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.enable_blob_files: false 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.min_blob_size: 0 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-14T17:15:17.577 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 809c628d-3af7-4219-a937-23e5ac6881cf 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707930917338336, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707930917338745, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707930917, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "809c628d-3af7-4219-a937-23e5ac6881cf", "db_session_id": "431WNKQ1510QUB9DM0NU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707930917338825, "job": 1, "event": "recovery_finished"} 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5579c3350000 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: DB pointer 0x5579c333a000 2024-02-14T17:15:17.578 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T17:15:17.579 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T17:15:17.579 INFO:journalctl@ceph.mon.c.smithi120.stdout: ** DB Stats ** 2024-02-14T17:15:17.579 INFO:journalctl@ceph.mon.c.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:15:17.579 INFO:journalctl@ceph.mon.c.smithi120.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T17:15:17.579 INFO:journalctl@ceph.mon.c.smithi120.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:15:17.580 INFO:journalctl@ceph.mon.c.smithi120.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:15:17.580 INFO:journalctl@ceph.mon.c.smithi120.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T17:15:17.580 INFO:journalctl@ceph.mon.c.smithi120.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:15:17.580 INFO:journalctl@ceph.mon.c.smithi120.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:15:17.580 INFO:journalctl@ceph.mon.c.smithi120.stdout: 2024-02-14T17:15:17.581 INFO:journalctl@ceph.mon.c.smithi120.stdout: ** Compaction Stats [default] ** 2024-02-14T17:15:17.581 INFO:journalctl@ceph.mon.c.smithi120.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:15:17.581 INFO:journalctl@ceph.mon.c.smithi120.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T17:15:17.581 INFO:journalctl@ceph.mon.c.smithi120.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:15:17.581 INFO:journalctl@ceph.mon.c.smithi120.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:15:17.581 INFO:journalctl@ceph.mon.c.smithi120.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:15:17.582 INFO:journalctl@ceph.mon.c.smithi120.stdout: 2024-02-14T17:15:17.582 INFO:journalctl@ceph.mon.c.smithi120.stdout: ** Compaction Stats [default] ** 2024-02-14T17:15:17.582 INFO:journalctl@ceph.mon.c.smithi120.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:15:17.582 INFO:journalctl@ceph.mon.c.smithi120.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T17:15:17.582 INFO:journalctl@ceph.mon.c.smithi120.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:15:17.583 INFO:journalctl@ceph.mon.c.smithi120.stdout: 2024-02-14T17:15:17.583 INFO:journalctl@ceph.mon.c.smithi120.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T17:15:17.583 INFO:journalctl@ceph.mon.c.smithi120.stdout: 2024-02-14T17:15:17.583 INFO:journalctl@ceph.mon.c.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:15:17.583 INFO:journalctl@ceph.mon.c.smithi120.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T17:15:17.583 INFO:journalctl@ceph.mon.c.smithi120.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T17:15:17.584 INFO:journalctl@ceph.mon.c.smithi120.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T17:15:17.584 INFO:journalctl@ceph.mon.c.smithi120.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T17:15:17.584 INFO:journalctl@ceph.mon.c.smithi120.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T17:15:17.584 INFO:journalctl@ceph.mon.c.smithi120.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:15:17.584 INFO:journalctl@ceph.mon.c.smithi120.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:15:17.584 INFO:journalctl@ceph.mon.c.smithi120.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout: Block cache BinnedLRUCache@0x5579c325f090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout: 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: using public_addr v2:172.21.15.120:0/0 -> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: starting mon.c rank -1 at public addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] at bind addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(???) e0 preinit fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).mds e1 new map 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout: e1 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T17:15:17.585 INFO:journalctl@ceph.mon.c.smithi120.stdout: legacy client fscid: -1 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout: 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout: No filesystems configured 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.586 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi091", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Updating smithi091:/etc/ceph/ceph.conf 2024-02-14T17:15:17.587 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Deploying cephadm binary to smithi120 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Updating smithi091:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Added host smithi120 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.588 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/798761070' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/798761070' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:15:17.589 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Updating smithi120:/etc/ceph/ceph.conf 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi091:172.21.15.91=b;smithi120:172.21.15.120=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi091:172.21.15.91=b;smithi120:172.21.15.120=c;count:3 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:17.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: Deploying daemon mon.c on smithi120 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: from='client.? 172.21.15.120:0/159723773' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 ceph-mon[30899]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 bash[30861]: 40bbe8467118cf800f0b81455edfb8d0a2e1fe7549c27195ca3ff39ef1e139c5 2024-02-14T17:15:17.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:17 smithi120 systemd[1]: Started Ceph mon.c for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:15:17.592 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:15:22.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:15:22.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: mon.a calling monitor election 2024-02-14T17:15:22.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:22.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: mon.c calling monitor election 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: monmap e2: 2 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:15:22.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: fsmap 2024-02-14T17:15:22.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:22.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: mgrmap e13: y(active, since 52s) 2024-02-14T17:15:22.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: overall HEALTH_OK 2024-02-14T17:15:22.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:22.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:22.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:22.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:22.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:22 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:22.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:15:22.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: mon.a calling monitor election 2024-02-14T17:15:22.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:22.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: mon.c calling monitor election 2024-02-14T17:15:22.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:22.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:22.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:22.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T17:15:22.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: monmap e2: 2 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:15:22.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: fsmap 2024-02-14T17:15:22.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:22.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: mgrmap e13: y(active, since 52s) 2024-02-14T17:15:22.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: overall HEALTH_OK 2024-02-14T17:15:22.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:22.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:22.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:22.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:22.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:22 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:22.946 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:15:22.946 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":2,"fsid":"39c4996e-cb5c-11ee-95b9-87774f69a715","modified":"2024-02-14T17:15:17.368648Z","created":"2024-02-14T17:13:24.964428Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-14T17:15:22.948 INFO:teuthology.orchestra.run.smithi120.stderr:dumped monmap epoch 2 2024-02-14T17:15:23.548 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:23 smithi120 ceph-mon[30899]: Deploying daemon mon.b on smithi091 2024-02-14T17:15:23.549 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:23 smithi120 ceph-mon[30899]: from='client.? 172.21.15.120:0/566319347' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:15:23.549 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:23 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:23.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:23 smithi082 ceph-mon[25796]: Deploying daemon mon.b on smithi091 2024-02-14T17:15:23.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:23 smithi082 ceph-mon[25796]: from='client.? 172.21.15.120:0/566319347' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:15:23.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:23 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:24.576 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:15:24.577 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:15:24.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:24 smithi082 ceph-mon[25796]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:24.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:24 smithi120 ceph-mon[30899]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:24.816 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:15:26.551 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 podman[30252]: 2024-02-14 17:15:26.373875499 +0000 UTC m=+0.017610056 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:15:26.568 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:15:26.569 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":2,"fsid":"39c4996e-cb5c-11ee-95b9-87774f69a715","modified":"2024-02-14T17:15:17.368648Z","created":"2024-02-14T17:13:24.964428Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-14T17:15:26.570 INFO:teuthology.orchestra.run.smithi120.stderr:dumped monmap epoch 2 2024-02-14T17:15:27.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 podman[30252]: 2024-02-14 17:15:26.623126641 +0000 UTC m=+0.266861199 container init d86e71bb3cb0698af58deaafbff0ae095af9ed592fad252411a13b659b4baca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-02-14T17:15:27.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 podman[30252]: 2024-02-14 17:15:26.628291567 +0000 UTC m=+0.272026132 container start d86e71bb3cb0698af58deaafbff0ae095af9ed592fad252411a13b659b4baca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-14T17:15:27.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T17:15:27.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev), process ceph-mon, pid 7 2024-02-14T17:15:27.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pidfile_write: ignore empty --pid-file 2024-02-14T17:15:27.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: load: jerasure load: lrc 2024-02-14T17:15:27.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: RocksDB version: 7.9.2 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Git sha 0 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Compile date 2024-02-14 11:59:27 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: DB SUMMARY 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: DB Session ID: Q77GB4QX3XGO3N1FTGRD 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: CURRENT file: CURRENT 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.error_if_exists: 0 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.create_if_missing: 0 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.paranoid_checks: 1 2024-02-14T17:15:27.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.env: 0x55c87d6d7ae0 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.info_log: 0x55c87fb458c0 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.statistics: (nil) 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.use_fsync: 0 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_log_file_size: 0 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T17:15:27.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.allow_fallocate: 1 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.use_direct_reads: 0 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.db_log_dir: 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.wal_dir: 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T17:15:27.107 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.write_buffer_manager: 0x55c87fb19400 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.unordered_write: 0 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T17:15:27.108 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T17:15:27.109 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T17:15:27.109 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.row_cache: None 2024-02-14T17:15:27.109 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.wal_filter: None 2024-02-14T17:15:27.110 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T17:15:27.110 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.two_write_queues: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.wal_compression: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.atomic_flush: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.log_readahead_size: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T17:15:27.111 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_background_jobs: 2 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_background_compactions: -1 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_subcompactions: 1 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T17:15:27.112 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_open_files: -1 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_background_flushes: -1 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Compression algorithms supported: 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: kZSTD supported: 0 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: kXpressCompression supported: 0 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: kZlibCompression supported: 1 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: kSnappyCompression supported: 1 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: kLZ4Compression supported: 1 2024-02-14T17:15:27.113 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: kBZip2Compression supported: 0 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.merge_operator: 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_filter: None 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c87fb459c0) 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout: cache_index_and_filter_blocks: 1 2024-02-14T17:15:27.114 INFO:journalctl@ceph.mon.b.smithi091.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: pin_top_level_index_and_filter: 1 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: index_type: 0 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: data_block_index_type: 0 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: index_shortening: 1 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: checksum: 4 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: no_block_cache: 0 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: block_cache: 0x55c87fb49090 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: block_cache_name: BinnedLRUCache 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: block_cache_options: 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: capacity : 536870912 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: num_shard_bits : 4 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: strict_capacity_limit : 0 2024-02-14T17:15:27.115 INFO:journalctl@ceph.mon.b.smithi091.stdout: high_pri_pool_ratio: 0.000 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: block_cache_compressed: (nil) 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: persistent_cache: (nil) 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: block_size: 4096 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: block_size_deviation: 10 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: block_restart_interval: 16 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: index_block_restart_interval: 1 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: metadata_block_size: 4096 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: partition_filters: 0 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: use_delta_encoding: 1 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: filter_policy: bloomfilter 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: whole_key_filtering: 1 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: verify_compression: 0 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: read_amp_bytes_per_bit: 0 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: format_version: 5 2024-02-14T17:15:27.116 INFO:journalctl@ceph.mon.b.smithi091.stdout: enable_index_compression: 1 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout: block_align: 0 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout: max_auto_readahead_size: 262144 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout: prepopulate_block_cache: 0 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout: initial_auto_readahead_size: 8192 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression: NoCompression 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.num_levels: 7 2024-02-14T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T17:15:27.118 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T17:15:27.119 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T17:15:27.120 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.table_properties_collectors: 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.inplace_update_support: 0 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.bloom_locality: 0 2024-02-14T17:15:27.121 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.max_successive_merges: 0 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.ttl: 2592000 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.enable_blob_files: false 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.min_blob_size: 0 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T17:15:27.122 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f3108543-3192-45cb-98bf-894d89a51607 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707930926658339, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707930926658745, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707930926, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f3108543-3192-45cb-98bf-894d89a51607", "db_session_id": "Q77GB4QX3XGO3N1FTGRD", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707930926658822, "job": 1, "event": "recovery_finished"} 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c87fc3a000 2024-02-14T17:15:27.123 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: DB pointer 0x55c87fc24000 2024-02-14T17:15:27.124 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T17:15:27.124 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T17:15:27.124 INFO:journalctl@ceph.mon.b.smithi091.stdout: ** DB Stats ** 2024-02-14T17:15:27.124 INFO:journalctl@ceph.mon.b.smithi091.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:15:27.124 INFO:journalctl@ceph.mon.b.smithi091.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T17:15:27.124 INFO:journalctl@ceph.mon.b.smithi091.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: ** Compaction Stats [default] ** 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:15:27.128 INFO:journalctl@ceph.mon.b.smithi091.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: ** Compaction Stats [default] ** 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T17:15:27.129 INFO:journalctl@ceph.mon.b.smithi091.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout: Cumulative compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout: Interval compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout: Block cache BinnedLRUCache@0x55c87fb49090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout: 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: using public_addr v2:172.21.15.91:0/0 -> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: starting mon.b rank -1 at public addrs [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] at bind addrs [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(???) e0 preinit fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:15:27.130 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).mds e1 new map 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout: e1 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout: legacy client fscid: -1 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout: 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout: No filesystems configured 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-14T17:15:27.131 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:27.132 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi091", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Updating smithi091:/etc/ceph/ceph.conf 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Deploying cephadm binary to smithi120 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Updating smithi091:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Added host smithi120 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/798761070' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T17:15:27.134 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/798761070' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Updating smithi120:/etc/ceph/ceph.conf 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:27.135 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi091:172.21.15.91=b;smithi120:172.21.15.120=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi091:172.21.15.91=b;smithi120:172.21.15.120=c;count:3 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Deploying daemon mon.c on smithi120 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.136 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='client.? 172.21.15.120:0/159723773' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.a calling monitor election 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.c calling monitor election 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.137 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: monmap e2: 2 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: fsmap 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mgrmap e13: y(active, since 52s) 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: overall HEALTH_OK 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: Deploying daemon mon.b on smithi091 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='client.? 172.21.15.120:0/566319347' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:27.138 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:27.139 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 ceph-mon[30290]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-14T17:15:27.139 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 bash[30252]: d86e71bb3cb0698af58deaafbff0ae095af9ed592fad252411a13b659b4baca8 2024-02-14T17:15:27.139 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:26 smithi091 systemd[1]: Started Ceph mon.b for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:15:28.184 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:15:28.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:15:28.425 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:15:32.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:15:32.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: mon.a calling monitor election 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: mon.c calling monitor election 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0],c=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: fsmap 2024-02-14T17:15:32.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:32.057 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: mgrmap e13: y(active, since 61s) 2024-02-14T17:15:32.057 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-14T17:15:32.057 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-14T17:15:32.057 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-14T17:15:32.057 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: mon.b (rank 2) addr [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] is down (out of quorum) 2024-02-14T17:15:32.057 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:32.057 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:31 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:15:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: mon.a calling monitor election 2024-02-14T17:15:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: mon.c calling monitor election 2024-02-14T17:15:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0],c=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: fsmap 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: mgrmap e13: y(active, since 61s) 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: mon.b (rank 2) addr [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] is down (out of quorum) 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:32.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:31 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:33.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:33.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:33.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:32 smithi120 ceph-mon[30899]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:33.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:33.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-02-14T17:15:33.126 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":3,"fsid":"39c4996e-cb5c-11ee-95b9-87774f69a715","modified":"2024-02-14T17:15:26.687040Z","created":"2024-02-14T17:13:24.964428Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:3300","nonce":0},{"type":"v1","addr":"172.21.15.91:6789","nonce":0}]},"addr":"172.21.15.91:6789/0","public_addr":"172.21.15.91:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-14T17:15:33.128 INFO:teuthology.orchestra.run.smithi120.stderr:dumped monmap epoch 3 2024-02-14T17:15:33.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:33.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:33.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:32 smithi082 ceph-mon[25796]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:33.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:33.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:33.931 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-14T17:15:33.931 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph config generate-minimal-conf 2024-02-14T17:15:34.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:15:34.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: mon.a calling monitor election 2024-02-14T17:15:34.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: mon.c calling monitor election 2024-02-14T17:15:34.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:34.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:15:34.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:34.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0],c=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: fsmap 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: mgrmap e13: y(active, since 61s) 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-14T17:15:34.105 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: mon.b (rank 2) addr [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] is down (out of quorum) 2024-02-14T17:15:34.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:34.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:34.106 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:34.179 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:15:34.569 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: mon.b calling monitor election 2024-02-14T17:15:34.569 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Updating smithi082:/etc/ceph/ceph.conf 2024-02-14T17:15:34.569 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Updating smithi091:/etc/ceph/ceph.conf 2024-02-14T17:15:34.569 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Updating smithi120:/etc/ceph/ceph.conf 2024-02-14T17:15:34.569 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:34.569 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: mon.b calling monitor election 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: mon.c calling monitor election 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: mon.a calling monitor election 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0],c=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: fsmap 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: mgrmap e13: y(active, since 63s) 2024-02-14T17:15:34.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Cluster is now healthy 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: overall HEALTH_OK 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.571 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T17:15:34.572 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:34.572 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:15:34.572 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:34.572 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:34 smithi082 ceph-mon[25796]: Reconfiguring daemon mon.a on smithi082 2024-02-14T17:15:34.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: mon.b calling monitor election 2024-02-14T17:15:34.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Updating smithi082:/etc/ceph/ceph.conf 2024-02-14T17:15:34.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Updating smithi091:/etc/ceph/ceph.conf 2024-02-14T17:15:34.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Updating smithi120:/etc/ceph/ceph.conf 2024-02-14T17:15:34.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:34.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:34.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:34.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: mon.b calling monitor election 2024-02-14T17:15:34.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: mon.c calling monitor election 2024-02-14T17:15:34.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: mon.a calling monitor election 2024-02-14T17:15:34.606 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-14T17:15:34.606 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0],c=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:15:34.606 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: fsmap 2024-02-14T17:15:34.606 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:34.606 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: mgrmap e13: y(active, since 63s) 2024-02-14T17:15:34.606 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-14T17:15:34.607 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Cluster is now healthy 2024-02-14T17:15:34.607 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: overall HEALTH_OK 2024-02-14T17:15:34.607 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.607 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.607 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.607 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.608 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.608 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.608 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.608 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:34.608 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.609 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.609 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.609 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.609 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T17:15:34.609 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:34.609 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:15:34.610 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:34.610 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:34 smithi091 ceph-mon[30290]: Reconfiguring daemon mon.a on smithi082 2024-02-14T17:15:34.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: mon.b calling monitor election 2024-02-14T17:15:34.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Updating smithi082:/etc/ceph/ceph.conf 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Updating smithi091:/etc/ceph/ceph.conf 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Updating smithi120:/etc/ceph/ceph.conf 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: mon.b calling monitor election 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: mon.c calling monitor election 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: mon.a calling monitor election 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0],c=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: fsmap 2024-02-14T17:15:34.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: mgrmap e13: y(active, since 63s) 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Cluster is now healthy 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: overall HEALTH_OK 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:34.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:34.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T17:15:34.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:34.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:15:34.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:34.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:34 smithi120 ceph-mon[30899]: Reconfiguring daemon mon.a on smithi082 2024-02-14T17:15:35.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:35 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:35.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:35 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:35.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:35 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:35.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:35 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:35.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:35 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:15:35.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:35 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:36.438 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:36 smithi082 ceph-mon[25796]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:36.447 INFO:teuthology.orchestra.run.smithi082.stdout:# minimal ceph.conf for 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:15:36.447 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-02-14T17:15:36.447 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:15:36.447 INFO:teuthology.orchestra.run.smithi082.stdout: mon_host = [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-02-14T17:15:36.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:36 smithi091 ceph-mon[30290]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:36.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:36 smithi120 ceph-mon[30899]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:37.354 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-14T17:15:37.354 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:15:37.354 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T17:15:37.383 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:15:37.383 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:15:37.450 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:15:37.451 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T17:15:37.498 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:15:37.498 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:15:37.566 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:15:37.566 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T17:15:37.595 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:37 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/1763798351' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:37.596 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:37 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:37.596 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:37 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:37.596 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:37 smithi120 ceph-mon[30899]: Reconfiguring mon.b (monmap changed)... 2024-02-14T17:15:37.596 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:37 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:37.596 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:37 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:15:37.596 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:37 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:37.596 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:37 smithi120 ceph-mon[30899]: Reconfiguring daemon mon.b on smithi091 2024-02-14T17:15:37.600 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:15:37.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:15:37.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:37 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/1763798351' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:37.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:37 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:37.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:37 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:37.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:37 smithi091 ceph-mon[30290]: Reconfiguring mon.b (monmap changed)... 2024-02-14T17:15:37.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:37 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:37.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:37 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:15:37.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:37 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:37.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:37 smithi091 ceph-mon[30290]: Reconfiguring daemon mon.b on smithi091 2024-02-14T17:15:37.671 INFO:tasks.cephadm:Adding mgr.y on smithi082 2024-02-14T17:15:37.671 INFO:tasks.cephadm:Adding mgr.x on smithi091 2024-02-14T17:15:37.671 INFO:tasks.cephadm:Adding mgr.z on smithi120 2024-02-14T17:15:37.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch apply mgr '3;smithi082=y;smithi091=x;smithi120=z' 2024-02-14T17:15:37.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:37 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1763798351' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:37 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:37 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:37 smithi082 ceph-mon[25796]: Reconfiguring mon.b (monmap changed)... 2024-02-14T17:15:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:37 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:37 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:15:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:37 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:37.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:37 smithi082 ceph-mon[25796]: Reconfiguring daemon mon.b on smithi091 2024-02-14T17:15:37.944 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:15:38.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:38 smithi091 ceph-mon[30290]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:38.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:38 smithi082 ceph-mon[25796]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:38.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:38 smithi120 ceph-mon[30899]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:39.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:39 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:39.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:39 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:39.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:39 smithi120 ceph-mon[30899]: Reconfiguring mon.c (monmap changed)... 2024-02-14T17:15:39.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:39 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:39.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:39 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:15:39.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:39 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:39.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:39 smithi120 ceph-mon[30899]: Reconfiguring daemon mon.c on smithi120 2024-02-14T17:15:39.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:39 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:39.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:39 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:39.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:39 smithi091 ceph-mon[30290]: Reconfiguring mon.c (monmap changed)... 2024-02-14T17:15:39.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:39 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:39.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:39 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:15:39.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:39 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:39.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:39 smithi091 ceph-mon[30290]: Reconfiguring daemon mon.c on smithi120 2024-02-14T17:15:39.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:39 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:39.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:39 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:39.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:39 smithi082 ceph-mon[25796]: Reconfiguring mon.c (monmap changed)... 2024-02-14T17:15:39.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:39 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:15:39.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:39 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:15:39.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:39 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:39.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:39 smithi082 ceph-mon[25796]: Reconfiguring daemon mon.c on smithi120 2024-02-14T17:15:40.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:40 smithi120 ceph-mon[30899]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:40.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:40 smithi091 ceph-mon[30290]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:40.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:40 smithi082 ceph-mon[25796]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:40.758 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled mgr update... 2024-02-14T17:15:41.329 DEBUG:teuthology.orchestra.run.smithi091:mgr.x> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.x.service 2024-02-14T17:15:41.332 DEBUG:teuthology.orchestra.run.smithi120:mgr.z> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.z.service 2024-02-14T17:15:41.334 INFO:tasks.cephadm:Deploying OSDs... 2024-02-14T17:15:41.334 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:15:41.334 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:15:41.350 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-14T17:15:41.351 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/nvme1n1 2024-02-14T17:15:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/nvme1n1 2024-02-14T17:15:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:41.406 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 764 Links: 1 Device type: 103,2 2024-02-14T17:15:41.406 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:41.406 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:41.406 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-14 17:14:42.855877934 +0000 2024-02-14T17:15:41.406 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-14 17:12:22.181569029 +0000 2024-02-14T17:15:41.406 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-14 17:12:22.181569029 +0000 2024-02-14T17:15:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-14 17:12:22.173569184 +0000 2024-02-14T17:15:41.406 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-14T17:15:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-14T17:15:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-14T17:15:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000234299 s, 2.2 MB/s 2024-02-14T17:15:41.473 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-14T17:15:41.529 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/nvme2n1 2024-02-14T17:15:41.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:41.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=y;smithi091=x;smithi120=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:41.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: Saving service mgr spec with placement smithi082=y;smithi091=x;smithi120=z;count:3 2024-02-14T17:15:41.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:41.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:41.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:15:41.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:15:41.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:15:41.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:41.557 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:41 smithi120 ceph-mon[30899]: Deploying daemon mgr.z on smithi120 2024-02-14T17:15:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/nvme2n1 2024-02-14T17:15:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:41.585 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-14T17:15:41.585 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:41.585 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:41.585 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-14 17:14:42.866877722 +0000 2024-02-14T17:15:41.585 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-14 17:12:22.343565896 +0000 2024-02-14T17:15:41.585 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-14 17:12:22.343565896 +0000 2024-02-14T17:15:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-14 17:12:22.331566128 +0000 2024-02-14T17:15:41.585 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-14T17:15:41.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:41.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=y;smithi091=x;smithi120=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:41.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: Saving service mgr spec with placement smithi082=y;smithi091=x;smithi120=z;count:3 2024-02-14T17:15:41.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:41.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:41.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:15:41.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:15:41.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:15:41.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:41.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:41 smithi082 ceph-mon[25796]: Deploying daemon mgr.z on smithi120 2024-02-14T17:15:41.650 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-14T17:15:41.650 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-14T17:15:41.650 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000192906 s, 2.7 MB/s 2024-02-14T17:15:41.652 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-14T17:15:41.707 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/nvme3n1 2024-02-14T17:15:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/nvme3n1 2024-02-14T17:15:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:41.762 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-14T17:15:41.762 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:41.762 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:41.762 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-14 17:14:42.879877472 +0000 2024-02-14T17:15:41.763 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-14 17:12:22.456563710 +0000 2024-02-14T17:15:41.763 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-14 17:12:22.456563710 +0000 2024-02-14T17:15:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-14 17:12:22.445563922 +0000 2024-02-14T17:15:41.763 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-14T17:15:41.828 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-14T17:15:41.828 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-14T17:15:41.828 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000204234 s, 2.5 MB/s 2024-02-14T17:15:41.829 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-14T17:15:41.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=y;smithi091=x;smithi120=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: Saving service mgr spec with placement smithi082=y;smithi091=x;smithi120=z;count:3 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:41.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:41 smithi091 ceph-mon[30290]: Deploying daemon mgr.z on smithi120 2024-02-14T17:15:41.885 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/nvme4n1 2024-02-14T17:15:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/nvme4n1 2024-02-14T17:15:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:41.941 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-14T17:15:41.941 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:41.941 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:41.941 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-14 17:14:42.890877260 +0000 2024-02-14T17:15:41.941 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-14 17:12:22.604560847 +0000 2024-02-14T17:15:41.941 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-14 17:12:22.604560847 +0000 2024-02-14T17:15:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-14 17:12:22.593561059 +0000 2024-02-14T17:15:41.941 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-14T17:15:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-14T17:15:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-14T17:15:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000255994 s, 2.0 MB/s 2024-02-14T17:15:42.008 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-14T17:15:42.064 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:15:42.064 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:15:42.083 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-14T17:15:42.083 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/nvme1n1 2024-02-14T17:15:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/nvme1n1 2024-02-14T17:15:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:42.141 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-14T17:15:42.141 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:42.141 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:42.141 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-14 17:14:57.894664080 +0000 2024-02-14T17:15:42.141 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-14 17:12:23.783551262 +0000 2024-02-14T17:15:42.141 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-14 17:12:23.783551262 +0000 2024-02-14T17:15:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-14 17:12:23.771551143 +0000 2024-02-14T17:15:42.142 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-14T17:15:42.211 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-14T17:15:42.211 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-14T17:15:42.211 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000300924 s, 1.7 MB/s 2024-02-14T17:15:42.212 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-14T17:15:42.270 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/nvme2n1 2024-02-14T17:15:42.327 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/nvme2n1 2024-02-14T17:15:42.327 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:42.327 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-14T17:15:42.327 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:42.327 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:42.327 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-14 17:14:57.905663804 +0000 2024-02-14T17:15:42.327 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-14 17:12:23.944552862 +0000 2024-02-14T17:15:42.327 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-14 17:12:23.944552862 +0000 2024-02-14T17:15:42.327 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-14 17:12:23.935552772 +0000 2024-02-14T17:15:42.327 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-14T17:15:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-14T17:15:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-14T17:15:42.396 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000300884 s, 1.7 MB/s 2024-02-14T17:15:42.397 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-14T17:15:42.454 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/nvme3n1 2024-02-14T17:15:42.510 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/nvme3n1 2024-02-14T17:15:42.510 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:42.510 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 786 Links: 1 Device type: 103,6 2024-02-14T17:15:42.510 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:42.510 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:42.510 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-14 17:14:57.916663528 +0000 2024-02-14T17:15:42.510 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-14 17:12:24.094554352 +0000 2024-02-14T17:15:42.511 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-14 17:12:24.094554352 +0000 2024-02-14T17:15:42.511 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-14 17:12:24.082554233 +0000 2024-02-14T17:15:42.511 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-14T17:15:42.574 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:42 smithi091 ceph-mon[30290]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:42.577 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-14T17:15:42.577 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-14T17:15:42.577 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000215218 s, 2.4 MB/s 2024-02-14T17:15:42.578 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-14T17:15:42.607 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:42 smithi120 ceph-mon[30899]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:42.634 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/nvme4n1 2024-02-14T17:15:42.689 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/nvme4n1 2024-02-14T17:15:42.689 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:42.689 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-14T17:15:42.689 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:42.690 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:42.690 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-14 17:14:57.926663277 +0000 2024-02-14T17:15:42.690 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-14 17:12:24.242555823 +0000 2024-02-14T17:15:42.690 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-14 17:12:24.242555823 +0000 2024-02-14T17:15:42.690 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-14 17:12:24.230555703 +0000 2024-02-14T17:15:42.690 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-14T17:15:42.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:42 smithi082 ceph-mon[25796]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:42.755 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-14T17:15:42.756 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-14T17:15:42.756 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000216596 s, 2.4 MB/s 2024-02-14T17:15:42.757 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-14T17:15:42.814 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:15:42.814 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:15:42.830 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-14T17:15:42.830 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/nvme1n1 2024-02-14T17:15:42.885 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/nvme1n1 2024-02-14T17:15:42.885 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:42.885 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-14T17:15:42.885 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:42.885 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:42.885 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-02-14 17:15:08.014667653 +0000 2024-02-14T17:15:42.885 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-02-14 17:12:25.427423844 +0000 2024-02-14T17:15:42.886 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-02-14 17:12:25.427423844 +0000 2024-02-14T17:15:42.886 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-02-14 17:12:25.415424121 +0000 2024-02-14T17:15:42.886 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-14T17:15:43.029 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-02-14T17:15:43.029 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-02-14T17:15:43.029 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000213505 s, 2.4 MB/s 2024-02-14T17:15:43.030 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-14T17:15:43.045 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/nvme2n1 2024-02-14T17:15:43.102 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/nvme2n1 2024-02-14T17:15:43.102 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:43.102 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 776 Links: 1 Device type: 103,4 2024-02-14T17:15:43.102 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:43.102 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:43.102 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-02-14 17:15:08.024667422 +0000 2024-02-14T17:15:43.102 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-02-14 17:12:25.591420059 +0000 2024-02-14T17:15:43.102 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-02-14 17:12:25.591420059 +0000 2024-02-14T17:15:43.102 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-02-14 17:12:25.581420289 +0000 2024-02-14T17:15:43.103 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-14T17:15:43.167 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:43 smithi120 systemd[1]: Starting Ceph mgr.z for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:15:43.171 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-02-14T17:15:43.171 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-02-14T17:15:43.171 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000197698 s, 2.6 MB/s 2024-02-14T17:15:43.172 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-14T17:15:43.227 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/nvme3n1 2024-02-14T17:15:43.283 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/nvme3n1 2024-02-14T17:15:43.283 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:43.283 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-14T17:15:43.283 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:43.283 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:43.283 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-02-14 17:15:08.035667167 +0000 2024-02-14T17:15:43.283 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-02-14 17:12:25.704417451 +0000 2024-02-14T17:15:43.283 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-02-14 17:12:25.704417451 +0000 2024-02-14T17:15:43.283 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-02-14 17:12:25.696417635 +0000 2024-02-14T17:15:43.284 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-14T17:15:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-02-14T17:15:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-02-14T17:15:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000229985 s, 2.2 MB/s 2024-02-14T17:15:43.350 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-14T17:15:43.408 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/nvme4n1 2024-02-14T17:15:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/nvme4n1 2024-02-14T17:15:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-14T17:15:43.464 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-14T17:15:43.464 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-14T17:15:43.464 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-14T17:15:43.464 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-02-14 17:15:08.046666912 +0000 2024-02-14T17:15:43.464 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-02-14 17:12:25.857413920 +0000 2024-02-14T17:15:43.464 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-02-14 17:12:25.857413920 +0000 2024-02-14T17:15:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-02-14 17:12:25.850414081 +0000 2024-02-14T17:15:43.464 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-14T17:15:43.513 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:43 smithi120 podman[33018]: 2024-02-14 17:15:43.160649798 +0000 UTC m=+0.018156761 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:15:43.514 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:43 smithi120 podman[33018]: 2024-02-14 17:15:43.276651013 +0000 UTC m=+0.134157968 container create c9754de8f5c2029a8e120e3f03b625ae4eb5f09ffebb6ab84531ae833896f5ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-14T17:15:43.514 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:43 smithi120 podman[33018]: 2024-02-14 17:15:43.513013581 +0000 UTC m=+0.370520540 container init c9754de8f5c2029a8e120e3f03b625ae4eb5f09ffebb6ab84531ae833896f5ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS) 2024-02-14T17:15:43.532 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-02-14T17:15:43.532 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-02-14T17:15:43.532 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000238846 s, 2.1 MB/s 2024-02-14T17:15:43.533 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-14T17:15:43.588 INFO:tasks.cephadm:Deploying osd.0 on smithi082 with /dev/nvme4n1... 2024-02-14T17:15:43.588 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-14T17:15:43.805 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:43 smithi120 podman[33018]: 2024-02-14 17:15:43.518981072 +0000 UTC m=+0.376488037 container start c9754de8f5c2029a8e120e3f03b625ae4eb5f09ffebb6ab84531ae833896f5ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-14T17:15:43.805 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:43 smithi120 ceph-mgr[33177]: -- 172.21.15.120:0/247973343 <== mon.1 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5613509be1a0 con 0x561350994c00 2024-02-14T17:15:43.805 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:43 smithi120 bash[33018]: c9754de8f5c2029a8e120e3f03b625ae4eb5f09ffebb6ab84531ae833896f5ba 2024-02-14T17:15:43.805 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:43 smithi120 systemd[1]: Started Ceph mgr.z for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:15:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:15:44.555 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:44.060+0000 7f8f4f6f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:15:44.555 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:44.153+0000 7f8f4f6f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:15:44.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:44 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:44 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:44 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:44 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:15:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:44 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:15:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:44 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:15:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:44 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:44 smithi091 ceph-mon[30290]: Deploying daemon mgr.x on smithi091 2024-02-14T17:15:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:44 smithi091 ceph-mon[30290]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:44.888 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:44.889 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:44.889 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:44.889 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:15:44.889 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:15:44.889 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:15:44.889 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:44 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:44.889 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:44 smithi082 ceph-mon[25796]: Deploying daemon mgr.x on smithi091 2024-02-14T17:15:44.889 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:44 smithi082 ceph-mon[25796]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:45.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:45.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:45.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:45.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:15:45.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:15:45.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:15:45.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:45.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-mon[30899]: Deploying daemon mgr.x on smithi091 2024-02-14T17:15:45.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-mon[30899]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:45.023 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:44.788+0000 7f8f4f6f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:15:45.023 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:44 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:44.881+0000 7f8f4f6f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:15:45.023 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:45 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:45.020+0000 7f8f4f6f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:15:45.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:45 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:45.114+0000 7f8f4f6f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:15:45.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:45 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:45.194+0000 7f8f4f6f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:15:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/nvme4n1 2024-02-14T17:15:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-14T17:15:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-14T17:15:45.707 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-02-14T17:15:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-02-14T17:15:45.707 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0149978 s, 699 MB/s 2024-02-14T17:15:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-02-14T17:15:45.737 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:45 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:45.451+0000 7f8f4f6f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:15:45.737 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:45 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:45.557+0000 7f8f4f6f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:15:45.737 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:45 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:45.642+0000 7f8f4f6f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:15:45.737 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:45 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:45.735+0000 7f8f4f6f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:15:46.014 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:45 smithi091 systemd[1]: Starting Ceph mgr.x for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:15:46.055 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:45 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:45.910+0000 7f8f4f6f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:15:46.265 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:46 smithi091 podman[31744]: 2024-02-14 17:15:46.01303884 +0000 UTC m=+0.105701853 container create e217a9aaa51d90e1aa11d933aee1446de61d9c3fefe2c0762dd7aeed037e404e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:15:46.265 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:46 smithi091 podman[31744]: 2024-02-14 17:15:45.925717828 +0000 UTC m=+0.018380857 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:15:46.265 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:46 smithi091 podman[31744]: 2024-02-14 17:15:46.26506263 +0000 UTC m=+0.357725647 container init e217a9aaa51d90e1aa11d933aee1446de61d9c3fefe2c0762dd7aeed037e404e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-14T17:15:46.337 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi082:/dev/nvme4n1 2024-02-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:15:46.603 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:46 smithi091 podman[31744]: 2024-02-14 17:15:46.270260135 +0000 UTC m=+0.362923150 container start e217a9aaa51d90e1aa11d933aee1446de61d9c3fefe2c0762dd7aeed037e404e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T17:15:46.603 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:46 smithi091 ceph-mgr[31783]: -- 172.21.15.91:0/3249070482 <== mon.2 v2:172.21.15.91:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561416f1e1a0 con 0x561416ef4800 2024-02-14T17:15:46.603 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:46 smithi091 bash[31744]: e217a9aaa51d90e1aa11d933aee1446de61d9c3fefe2c0762dd7aeed037e404e 2024-02-14T17:15:46.603 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:46 smithi091 systemd[1]: Started Ceph mgr.x for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:15:47.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:46 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:46.828+0000 7f18ed447200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:15:47.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:46 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:46.924+0000 7f18ed447200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:15:47.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-mon[30290]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:47.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:47.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:47 smithi082 ceph-mon[25796]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:47.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:47 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:47 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:47 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:47 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:47 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:47.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:46 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:46.892+0000 7f8f4f6f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:15:47.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:47.030+0000 7f8f4f6f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:15:47.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-mon[30899]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:47.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:47.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:47.694 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:47.393+0000 7f8f4f6f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:15:47.694 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:47.488+0000 7f8f4f6f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:15:47.694 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:47.692+0000 7f8f4f6f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:15:47.709 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:47.613+0000 7f18ed447200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:15:47.709 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:47.707+0000 7f18ed447200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:15:48.038 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:47.850+0000 7f18ed447200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:15:48.038 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:47 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:47.951+0000 7f18ed447200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:15:48.038 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:48.036+0000 7f18ed447200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:15:48.055 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:47 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:47.879+0000 7f8f4f6f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:15:48.426 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:48.169+0000 7f8f4f6f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:15:48.426 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:48.424+0000 7f8f4f6f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:15:48.588 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:48.299+0000 7f18ed447200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:15:48.588 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:48.408+0000 7f18ed447200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:15:48.588 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:48.493+0000 7f18ed447200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:15:48.588 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:48.586+0000 7f18ed447200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:15:48.805 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:48.656+0000 7f8f4f6f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:15:48.853 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:48.761+0000 7f18ed447200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: Reconfiguring mgr.y (unknown last config time)... 2024-02-14T17:15:48.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:15:48.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:15:48.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:48.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: Reconfiguring daemon mgr.y on smithi082 2024-02-14T17:15:48.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:15:48.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:15:48.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:49.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:49.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: Reconfiguring mgr.y (unknown last config time)... 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: Reconfiguring daemon mgr.y on smithi082 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:15:49.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:49.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: Reconfiguring mgr.y (unknown last config time)... 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: Reconfiguring daemon mgr.y on smithi082 2024-02-14T17:15:49.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:15:49.355 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:15:49.355 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:49.805 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:49 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:49.443+0000 7f8f4f6f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:15:49.941 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:49 smithi082 ceph-mon[25796]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:49.941 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:49 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.941 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:49 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:49.941 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:49 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:49.941 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:49 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:49.941 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:49 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:49.941 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:49 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:50.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:49 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:49.721+0000 7f18ed447200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:15:50.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:49 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:49.853+0000 7f18ed447200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:15:50.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:49 smithi091 ceph-mon[30290]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:50.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:49 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:50.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:49 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:50.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:49 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:50.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:49 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:50.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:49 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:50.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:49 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:50.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:50 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:50.011+0000 7f8f4f6f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:15:50.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:50 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:50.107+0000 7f8f4f6f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:15:50.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:49 smithi120 ceph-mon[30899]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:15:50.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:49 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:50.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:49 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:50.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:49 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:15:50.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:49 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:50.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:49 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:15:50.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:49 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:50.467 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:50 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:50.185+0000 7f18ed447200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:15:50.467 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:50 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:50.273+0000 7f18ed447200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:15:50.467 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:50 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:50.465+0000 7f18ed447200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:15:50.805 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:15:50 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:15:50.320+0000 7f8f4f6f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:15:50.853 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:50 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:50.646+0000 7f18ed447200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:15:51.102 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:51 smithi082 ceph-mon[25796]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:51.102 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:51 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:51.103 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:51 smithi082 ceph-mon[25796]: Standby manager daemon z started 2024-02-14T17:15:51.103 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:51 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T17:15:51.103 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:51 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:15:51.103 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:51 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T17:15:51.103 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:51 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:15:51.185 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:51 smithi091 ceph-mon[30290]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:51.185 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:51 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:51.185 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:51 smithi091 ceph-mon[30290]: Standby manager daemon z started 2024-02-14T17:15:51.185 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:51 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T17:15:51.185 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:51 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:15:51.185 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:51 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T17:15:51.185 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:51 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:15:51.186 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:50 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:50.926+0000 7f18ed447200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:15:51.186 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:51 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:51.183+0000 7f18ed447200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:15:51.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:51 smithi120 ceph-mon[30899]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:51.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:51 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:15:51.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:51 smithi120 ceph-mon[30899]: Standby manager daemon z started 2024-02-14T17:15:51.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:51 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T17:15:51.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:51 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:15:51.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:51 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T17:15:51.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:51 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.120:0/1514763770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:15:51.603 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:51 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:51.416+0000 7f18ed447200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:15:52.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:52 smithi120 ceph-mon[30899]: mgrmap e14: y(active, since 81s), standbys: z 2024-02-14T17:15:52.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:52 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T17:15:52.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:52 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/470328885' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94a14f4e-e88f-4208-87ed-78b6945d4d47"}]: dispatch 2024-02-14T17:15:52.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:52 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/470328885' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94a14f4e-e88f-4208-87ed-78b6945d4d47"}]': finished 2024-02-14T17:15:52.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:52 smithi120 ceph-mon[30899]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T17:15:52.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:52 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:15:52.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:52 smithi082 ceph-mon[25796]: mgrmap e14: y(active, since 81s), standbys: z 2024-02-14T17:15:52.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:52 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T17:15:52.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:52 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/470328885' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94a14f4e-e88f-4208-87ed-78b6945d4d47"}]: dispatch 2024-02-14T17:15:52.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:52 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/470328885' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94a14f4e-e88f-4208-87ed-78b6945d4d47"}]': finished 2024-02-14T17:15:52.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:52 smithi082 ceph-mon[25796]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T17:15:52.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:52 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:15:52.353 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:52 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:52.212+0000 7f18ed447200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:15:52.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:52 smithi091 ceph-mon[30290]: mgrmap e14: y(active, since 81s), standbys: z 2024-02-14T17:15:52.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:52 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T17:15:52.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:52 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/470328885' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94a14f4e-e88f-4208-87ed-78b6945d4d47"}]: dispatch 2024-02-14T17:15:52.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:52 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/470328885' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94a14f4e-e88f-4208-87ed-78b6945d4d47"}]': finished 2024-02-14T17:15:52.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:52 smithi091 ceph-mon[30290]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T17:15:52.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:52 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:15:53.085 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:53 smithi091 ceph-mon[30290]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:53.085 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:52 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:52.775+0000 7f18ed447200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:15:53.085 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:52 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:52.870+0000 7f18ed447200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:15:53.085 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:15:53 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:15:53.083+0000 7f18ed447200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:15:53.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:53 smithi082 ceph-mon[25796]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:53.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:53 smithi120 ceph-mon[30899]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:54.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:54 smithi091 ceph-mon[30290]: Standby manager daemon x started 2024-02-14T17:15:54.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:54 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:15:54.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:54 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:15:54.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:54 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:15:54.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:54 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:15:54.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:54 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2310095718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:15:54.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:54 smithi120 ceph-mon[30899]: Standby manager daemon x started 2024-02-14T17:15:54.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:54 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:15:54.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:54 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:15:54.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:54 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:15:54.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:54 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:15:54.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:54 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2310095718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:15:54.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:54 smithi082 ceph-mon[25796]: Standby manager daemon x started 2024-02-14T17:15:54.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:54 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:15:54.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:54 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:15:54.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:54 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:15:54.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:54 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.91:0/3881592576' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:15:54.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:54 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2310095718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:15:55.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:55 smithi120 ceph-mon[30899]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:55.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:55 smithi120 ceph-mon[30899]: mgrmap e15: y(active, since 84s), standbys: z, x 2024-02-14T17:15:55.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:55 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:15:55.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:55 smithi091 ceph-mon[30290]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:55.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:55 smithi091 ceph-mon[30290]: mgrmap e15: y(active, since 84s), standbys: z, x 2024-02-14T17:15:55.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:55 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:15:55.421 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:55 smithi082 ceph-mon[25796]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:55.422 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:55 smithi082 ceph-mon[25796]: mgrmap e15: y(active, since 84s), standbys: z, x 2024-02-14T17:15:55.422 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:55 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:15:57.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:57 smithi082 ceph-mon[25796]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:57.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:57 smithi120 ceph-mon[30899]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:57.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:57 smithi091 ceph-mon[30290]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:59.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:59 smithi120 ceph-mon[30899]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:59.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:59 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T17:15:59.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:15:59 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:59.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:59 smithi091 ceph-mon[30290]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:59.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:59 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T17:15:59.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:15:59 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:15:59.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:59 smithi082 ceph-mon[25796]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:15:59.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:59 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T17:15:59.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:15:59 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:00.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:00 smithi082 ceph-mon[25796]: Deploying daemon osd.0 on smithi082 2024-02-14T17:16:00.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:00 smithi120 ceph-mon[30899]: Deploying daemon osd.0 on smithi082 2024-02-14T17:16:00.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:00 smithi091 ceph-mon[30290]: Deploying daemon osd.0 on smithi082 2024-02-14T17:16:01.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:01 smithi082 ceph-mon[25796]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:01.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:01 smithi120 ceph-mon[30899]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:01.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:01 smithi091 ceph-mon[30290]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:03.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:03 smithi120 ceph-mon[30899]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:03.346 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:03 smithi082 ceph-mon[25796]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:03.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:03 smithi091 ceph-mon[30290]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:05.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:05 smithi120 ceph-mon[30899]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:05.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:05.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:05.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:05.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:05 smithi091 ceph-mon[30290]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:05.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:05.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:05.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:05.364 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:05 smithi082 ceph-mon[25796]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:05.365 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:05.365 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:05.365 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:07.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:07 smithi120 ceph-mon[30899]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:07.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:07 smithi091 ceph-mon[30290]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:07.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:07 smithi082 ceph-mon[25796]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:08.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:08 smithi120 ceph-mon[30899]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:16:08.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:08 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:08.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:08 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:08.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:08 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:08.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:08 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:08.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:08 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:08.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:08 smithi091 ceph-mon[30290]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:16:08.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:08 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:08.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:08 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:08.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:08 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:08.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:08 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:08.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:08 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:08.448 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:08 smithi082 ceph-mon[25796]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:16:08.449 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:08 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:08.449 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:08 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:08.449 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:08 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:08.449 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:08 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:08.449 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:08 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:08.780 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 0 on host 'smithi082' 2024-02-14T17:16:09.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-mon[25796]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:09.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-mon[25796]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T17:16:09.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-mon[25796]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T17:16:09.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-mon[25796]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-02-14T17:16:09.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:09.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:09.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:09.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:09.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:09 smithi091 ceph-mon[30290]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:09.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:09 smithi091 ceph-mon[30290]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T17:16:09.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:09 smithi091 ceph-mon[30290]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T17:16:09.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:09 smithi091 ceph-mon[30290]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-02-14T17:16:09.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:09 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:09.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:09 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:09.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:09 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:09.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:09 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:09.377 DEBUG:teuthology.orchestra.run.smithi082:osd.0> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.0.service 2024-02-14T17:16:09.380 INFO:tasks.cephadm:Deploying osd.1 on smithi091 with /dev/nvme4n1... 2024-02-14T17:16:09.380 DEBUG:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-14T17:16:09.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:09 smithi120 ceph-mon[30899]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:09.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:09 smithi120 ceph-mon[30899]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T17:16:09.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:09 smithi120 ceph-mon[30899]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T17:16:09.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:09 smithi120 ceph-mon[30899]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-02-14T17:16:09.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:09 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:09.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:09 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:09.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:09 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:09.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:09 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:09.623 INFO:teuthology.orchestra.run.smithi091.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:16:09.678 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0[44024]: 2024-02-14T17:16:09.522+0000 7fc87e2cb700 -1 osd.0 0 waiting for initial osdmap 2024-02-14T17:16:09.678 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:16:09 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0[44024]: 2024-02-14T17:16:09.525+0000 7fc8760b6700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:16:10.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:10 smithi082 ceph-mon[25796]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-02-14T17:16:10.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:10 smithi082 ceph-mon[25796]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T17:16:10.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:10 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:10.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:10 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:10.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:10 smithi082 ceph-mon[25796]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' 2024-02-14T17:16:10.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:10 smithi091 ceph-mon[30290]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-02-14T17:16:10.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:10 smithi091 ceph-mon[30290]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T17:16:10.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:10 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:10.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:10 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:10.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:10 smithi091 ceph-mon[30290]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' 2024-02-14T17:16:10.456 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:10 smithi120 ceph-mon[30899]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-02-14T17:16:10.456 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:10 smithi120 ceph-mon[30899]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T17:16:10.456 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:10 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:10.456 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:10 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:10.457 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:10 smithi120 ceph-mon[30899]: from='osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684]' entity='osd.0' 2024-02-14T17:16:11.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:11 smithi082 ceph-mon[25796]: purged_snaps scrub starts 2024-02-14T17:16:11.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:11 smithi082 ceph-mon[25796]: purged_snaps scrub ok 2024-02-14T17:16:11.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:11 smithi082 ceph-mon[25796]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:11.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:11 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:11.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:11 smithi082 ceph-mon[25796]: osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684] boot 2024-02-14T17:16:11.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:11 smithi082 ceph-mon[25796]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T17:16:11.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:11 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:11.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:11 smithi120 ceph-mon[30899]: purged_snaps scrub starts 2024-02-14T17:16:11.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:11 smithi120 ceph-mon[30899]: purged_snaps scrub ok 2024-02-14T17:16:11.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:11 smithi120 ceph-mon[30899]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:11.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:11 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:11.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:11 smithi120 ceph-mon[30899]: osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684] boot 2024-02-14T17:16:11.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:11 smithi120 ceph-mon[30899]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T17:16:11.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:11 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:11.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:11 smithi091 ceph-mon[30290]: purged_snaps scrub starts 2024-02-14T17:16:11.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:11 smithi091 ceph-mon[30290]: purged_snaps scrub ok 2024-02-14T17:16:11.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:11 smithi091 ceph-mon[30290]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:16:11.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:11 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:11.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:11 smithi091 ceph-mon[30290]: osd.0 [v2:172.21.15.82:6802/1287971684,v1:172.21.15.82:6803/1287971684] boot 2024-02-14T17:16:11.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:11 smithi091 ceph-mon[30290]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T17:16:11.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:11 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:16:11.444 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping: /dev/nvme4n1 2024-02-14T17:16:11.444 INFO:teuthology.orchestra.run.smithi091.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-14T17:16:11.444 INFO:teuthology.orchestra.run.smithi091.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-14T17:16:11.444 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: 10+0 records in 2024-02-14T17:16:11.444 INFO:teuthology.orchestra.run.smithi091.stderr:10+0 records out 2024-02-14T17:16:11.445 INFO:teuthology.orchestra.run.smithi091.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125242 s, 837 MB/s 2024-02-14T17:16:11.445 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping successful for: 2024-02-14T17:16:12.045 DEBUG:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi091:/dev/nvme4n1 2024-02-14T17:16:12.277 INFO:teuthology.orchestra.run.smithi091.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:16:12.638 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:12 smithi091 ceph-mon[30290]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T17:16:12.639 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:12 smithi091 ceph-mon[30290]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:12.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:12 smithi120 ceph-mon[30899]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T17:16:12.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:12 smithi120 ceph-mon[30899]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:12.881 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:12 smithi082 ceph-mon[25796]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T17:16:12.881 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:12 smithi082 ceph-mon[25796]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:13.833 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:13 smithi091 ceph-mon[30290]: Detected new or changed devices on smithi082 2024-02-14T17:16:13.833 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:13 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:13.833 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:13 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:13.833 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:13 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:16:13.834 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:13 smithi091 ceph-mon[30290]: Adjusting osd_memory_target on smithi082 to 17088M 2024-02-14T17:16:13.834 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:13 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:13.834 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:13 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:13.834 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:13 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:13.834 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:13 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:13.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:13 smithi082 ceph-mon[25796]: Detected new or changed devices on smithi082 2024-02-14T17:16:13.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:13 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:13.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:13 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:13.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:13 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:16:13.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:13 smithi082 ceph-mon[25796]: Adjusting osd_memory_target on smithi082 to 17088M 2024-02-14T17:16:13.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:13 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:13.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:13 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:13.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:13 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:13.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:13 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:14.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:13 smithi120 ceph-mon[30899]: Detected new or changed devices on smithi082 2024-02-14T17:16:14.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:13 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:14.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:13 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:14.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:13 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:16:14.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:13 smithi120 ceph-mon[30899]: Adjusting osd_memory_target on smithi082 to 17088M 2024-02-14T17:16:14.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:13 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:14.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:13 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:14.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:13 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:14.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:13 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:14.939 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:14 smithi091 ceph-mon[30290]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:14.939 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:14 smithi091 ceph-mon[30290]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi091:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:16:14.939 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:14 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:16:14.939 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:14 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:16:14.939 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:14 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:14.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:14 smithi082 ceph-mon[25796]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:14.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:14 smithi082 ceph-mon[25796]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi091:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:16:14.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:14 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:16:14.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:14 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:16:14.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:14 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:15.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:14 smithi120 ceph-mon[30899]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:15.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:14 smithi120 ceph-mon[30899]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi091:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:16:15.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:14 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:16:15.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:14 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:16:15.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:14 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:17.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:17 smithi120 ceph-mon[30899]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:17.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:17 smithi120 ceph-mon[30899]: from='client.? 172.21.15.91:0/166331598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b3cadcf-e50d-453b-8b69-ed6fd1218d51"}]: dispatch 2024-02-14T17:16:17.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:17 smithi120 ceph-mon[30899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b3cadcf-e50d-453b-8b69-ed6fd1218d51"}]: dispatch 2024-02-14T17:16:17.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:17 smithi120 ceph-mon[30899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b3cadcf-e50d-453b-8b69-ed6fd1218d51"}]': finished 2024-02-14T17:16:17.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:17 smithi120 ceph-mon[30899]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T17:16:17.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:17 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:17.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:17 smithi091 ceph-mon[30290]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:17.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:17 smithi091 ceph-mon[30290]: from='client.? 172.21.15.91:0/166331598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b3cadcf-e50d-453b-8b69-ed6fd1218d51"}]: dispatch 2024-02-14T17:16:17.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:17 smithi091 ceph-mon[30290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b3cadcf-e50d-453b-8b69-ed6fd1218d51"}]: dispatch 2024-02-14T17:16:17.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:17 smithi091 ceph-mon[30290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b3cadcf-e50d-453b-8b69-ed6fd1218d51"}]': finished 2024-02-14T17:16:17.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:17 smithi091 ceph-mon[30290]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T17:16:17.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:17 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:17.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:17 smithi082 ceph-mon[25796]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:17.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:17 smithi082 ceph-mon[25796]: from='client.? 172.21.15.91:0/166331598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b3cadcf-e50d-453b-8b69-ed6fd1218d51"}]: dispatch 2024-02-14T17:16:17.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:17 smithi082 ceph-mon[25796]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b3cadcf-e50d-453b-8b69-ed6fd1218d51"}]: dispatch 2024-02-14T17:16:17.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:17 smithi082 ceph-mon[25796]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b3cadcf-e50d-453b-8b69-ed6fd1218d51"}]': finished 2024-02-14T17:16:17.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:17 smithi082 ceph-mon[25796]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T17:16:17.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:17 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:18.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:18 smithi120 ceph-mon[30899]: from='client.? 172.21.15.91:0/2108931714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:16:18.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:18 smithi091 ceph-mon[30290]: from='client.? 172.21.15.91:0/2108931714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:16:18.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:18 smithi082 ceph-mon[25796]: from='client.? 172.21.15.91:0/2108931714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:16:19.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:19 smithi120 ceph-mon[30899]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:19.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:19 smithi091 ceph-mon[30290]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:19.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:19 smithi082 ceph-mon[25796]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:21.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:21 smithi120 ceph-mon[30899]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:21.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:21 smithi091 ceph-mon[30290]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:21.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:21 smithi082 ceph-mon[25796]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:23.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:23 smithi091 ceph-mon[30290]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:23.280 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:23 smithi082 ceph-mon[25796]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:23.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:23 smithi120 ceph-mon[30899]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:24.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:24 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T17:16:24.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:24 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:24.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:24 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T17:16:24.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:24 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:24.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:24 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T17:16:24.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:24 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:25.233 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:25 smithi091 ceph-mon[30290]: Deploying daemon osd.1 on smithi091 2024-02-14T17:16:25.233 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:25 smithi091 ceph-mon[30290]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:25.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:25 smithi120 ceph-mon[30899]: Deploying daemon osd.1 on smithi091 2024-02-14T17:16:25.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:25 smithi120 ceph-mon[30899]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:25.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:25 smithi082 ceph-mon[25796]: Deploying daemon osd.1 on smithi091 2024-02-14T17:16:25.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:25 smithi082 ceph-mon[25796]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:27.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:27 smithi120 ceph-mon[30899]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:27.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:27 smithi091 ceph-mon[30290]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:27.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:27 smithi082 ceph-mon[25796]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:29.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:29 smithi120 ceph-mon[30899]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:29.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:29 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:29.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:29 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:29.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:29 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:29.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:29 smithi091 ceph-mon[30290]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:29.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:29 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:29.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:29 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:29.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:29 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:29.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:29 smithi082 ceph-mon[25796]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:29.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:29.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:29.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:29 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:31.233 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:31 smithi091 ceph-mon[30290]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:31.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:31 smithi120 ceph-mon[30899]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:31 smithi082 ceph-mon[25796]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:32.529 INFO:teuthology.orchestra.run.smithi091.stdout:Created osd(s) 1 on host 'smithi091' 2024-02-14T17:16:32.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:32.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:32.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:32.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='osd.1 [v2:172.21.15.91:6800/881479687,v1:172.21.15.91:6801/881479687]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:16:32.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:16:32.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:32 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:32.845 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.845 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.845 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:32.845 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:32.845 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.846 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:32.846 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='osd.1 [v2:172.21.15.91:6800/881479687,v1:172.21.15.91:6801/881479687]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:16:32.846 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:16:32.846 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.846 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.846 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:32 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='osd.1 [v2:172.21.15.91:6800/881479687,v1:172.21.15.91:6801/881479687]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:32.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:32 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:33.157 DEBUG:teuthology.orchestra.run.smithi091:osd.1> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.1.service 2024-02-14T17:16:33.159 INFO:tasks.cephadm:Deploying osd.2 on smithi120 with /dev/nvme4n1... 2024-02-14T17:16:33.160 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-14T17:16:33.405 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:16:33.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:33 smithi120 ceph-mon[30899]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T17:16:33.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:33 smithi120 ceph-mon[30899]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T17:16:33.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:33 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:33.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:33 smithi120 ceph-mon[30899]: from='osd.1 [v2:172.21.15.91:6800/881479687,v1:172.21.15.91:6801/881479687]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi091", "root=default"]}]: dispatch 2024-02-14T17:16:33.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:33 smithi120 ceph-mon[30899]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi091", "root=default"]}]: dispatch 2024-02-14T17:16:33.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:33 smithi091 ceph-mon[30290]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T17:16:33.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:33 smithi091 ceph-mon[30290]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T17:16:33.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:33 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:33.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:33 smithi091 ceph-mon[30290]: from='osd.1 [v2:172.21.15.91:6800/881479687,v1:172.21.15.91:6801/881479687]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi091", "root=default"]}]: dispatch 2024-02-14T17:16:33.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:33 smithi091 ceph-mon[30290]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi091", "root=default"]}]: dispatch 2024-02-14T17:16:33.854 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:16:33 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1[34858]: 2024-02-14T17:16:33.540+0000 7fdcf5bce700 -1 osd.1 0 waiting for initial osdmap 2024-02-14T17:16:33.855 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:16:33 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1[34858]: 2024-02-14T17:16:33.542+0000 7fdcf19c1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:16:33.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:33 smithi082 ceph-mon[25796]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T17:16:33.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:33 smithi082 ceph-mon[25796]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T17:16:33.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:33 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:33.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:33 smithi082 ceph-mon[25796]: from='osd.1 [v2:172.21.15.91:6800/881479687,v1:172.21.15.91:6801/881479687]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi091", "root=default"]}]: dispatch 2024-02-14T17:16:33.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:33 smithi082 ceph-mon[25796]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi091", "root=default"]}]: dispatch 2024-02-14T17:16:34.548 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:34 smithi091 ceph-mon[30290]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi091", "root=default"]}]': finished 2024-02-14T17:16:34.548 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:34 smithi091 ceph-mon[30290]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T17:16:34.548 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:34.548 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:34 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:34.548 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:34 smithi091 ceph-mon[30290]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:34.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:34 smithi120 ceph-mon[30899]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi091", "root=default"]}]': finished 2024-02-14T17:16:34.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:34 smithi120 ceph-mon[30899]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T17:16:34.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:34.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:34 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:34.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:34 smithi120 ceph-mon[30899]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:34.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:34 smithi082 ceph-mon[25796]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi091", "root=default"]}]': finished 2024-02-14T17:16:34.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:34 smithi082 ceph-mon[25796]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T17:16:34.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:34.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:34 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:34.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:34 smithi082 ceph-mon[25796]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:16:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/nvme4n1 2024-02-14T17:16:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-14T17:16:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-14T17:16:35.383 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-02-14T17:16:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-02-14T17:16:35.383 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124797 s, 840 MB/s 2024-02-14T17:16:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-02-14T17:16:35.693 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:35 smithi120 ceph-mon[30899]: purged_snaps scrub starts 2024-02-14T17:16:35.693 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:35 smithi120 ceph-mon[30899]: purged_snaps scrub ok 2024-02-14T17:16:35.693 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:35 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:35.693 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:35 smithi120 ceph-mon[30899]: osd.1 [v2:172.21.15.91:6800/881479687,v1:172.21.15.91:6801/881479687] boot 2024-02-14T17:16:35.693 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:35 smithi120 ceph-mon[30899]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T17:16:35.694 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:35 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:35.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:35 smithi091 ceph-mon[30290]: purged_snaps scrub starts 2024-02-14T17:16:35.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:35 smithi091 ceph-mon[30290]: purged_snaps scrub ok 2024-02-14T17:16:35.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:35 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:35.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:35 smithi091 ceph-mon[30290]: osd.1 [v2:172.21.15.91:6800/881479687,v1:172.21.15.91:6801/881479687] boot 2024-02-14T17:16:35.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:35 smithi091 ceph-mon[30290]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T17:16:35.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:35 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:35.963 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi120:/dev/nvme4n1 2024-02-14T17:16:35.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:35 smithi082 ceph-mon[25796]: purged_snaps scrub starts 2024-02-14T17:16:35.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:35 smithi082 ceph-mon[25796]: purged_snaps scrub ok 2024-02-14T17:16:35.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:35 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:35.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:35 smithi082 ceph-mon[25796]: osd.1 [v2:172.21.15.91:6800/881479687,v1:172.21.15.91:6801/881479687] boot 2024-02-14T17:16:35.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:35 smithi082 ceph-mon[25796]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T17:16:35.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:35 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:16:36.194 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:16:36.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:36 smithi120 ceph-mon[30899]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T17:16:36.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:36 smithi120 ceph-mon[30899]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:36.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:16:36.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:36.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:36.563 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:36 smithi091 ceph-mon[30290]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T17:16:36.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:36 smithi091 ceph-mon[30290]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:36.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:16:36.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:36.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:36.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:36 smithi082 ceph-mon[25796]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T17:16:36.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:36 smithi082 ceph-mon[25796]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:36.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:16:36.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:36.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:36.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:36.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:37.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:37 smithi120 ceph-mon[30899]: Detected new or changed devices on smithi091 2024-02-14T17:16:37.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:37 smithi120 ceph-mon[30899]: Adjusting osd_memory_target on smithi091 to 17088M 2024-02-14T17:16:37.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:37 smithi091 ceph-mon[30290]: Detected new or changed devices on smithi091 2024-02-14T17:16:37.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:37 smithi091 ceph-mon[30290]: Adjusting osd_memory_target on smithi091 to 17088M 2024-02-14T17:16:37.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:37 smithi082 ceph-mon[25796]: Detected new or changed devices on smithi091 2024-02-14T17:16:37.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:37 smithi082 ceph-mon[25796]: Adjusting osd_memory_target on smithi091 to 17088M 2024-02-14T17:16:38.764 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:38 smithi120 ceph-mon[30899]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:16:38.764 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:38 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:16:38.765 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:38 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:16:38.765 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:38 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:38.765 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:38 smithi120 ceph-mon[30899]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:38.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:38 smithi091 ceph-mon[30290]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:16:38.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:38 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:16:38.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:38 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:16:38.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:38 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:38.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:38 smithi091 ceph-mon[30290]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:38.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:38 smithi082 ceph-mon[25796]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:16:38.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:38 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:16:38.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:38 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:16:38.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:38 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:38.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:38 smithi082 ceph-mon[25796]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:41.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:41 smithi120 ceph-mon[30899]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:41.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:41 smithi120 ceph-mon[30899]: from='client.? 172.21.15.120:0/3384848799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f93c6d5-4d0a-4a81-9e56-931c78ec163d"}]: dispatch 2024-02-14T17:16:41.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:41 smithi120 ceph-mon[30899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f93c6d5-4d0a-4a81-9e56-931c78ec163d"}]: dispatch 2024-02-14T17:16:41.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:41 smithi120 ceph-mon[30899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f93c6d5-4d0a-4a81-9e56-931c78ec163d"}]': finished 2024-02-14T17:16:41.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:41 smithi120 ceph-mon[30899]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T17:16:41.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:41 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:41.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:41 smithi091 ceph-mon[30290]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:41.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:41 smithi091 ceph-mon[30290]: from='client.? 172.21.15.120:0/3384848799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f93c6d5-4d0a-4a81-9e56-931c78ec163d"}]: dispatch 2024-02-14T17:16:41.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:41 smithi091 ceph-mon[30290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f93c6d5-4d0a-4a81-9e56-931c78ec163d"}]: dispatch 2024-02-14T17:16:41.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:41 smithi091 ceph-mon[30290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f93c6d5-4d0a-4a81-9e56-931c78ec163d"}]': finished 2024-02-14T17:16:41.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:41 smithi091 ceph-mon[30290]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T17:16:41.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:41 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:41.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:41 smithi082 ceph-mon[25796]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:41.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:41 smithi082 ceph-mon[25796]: from='client.? 172.21.15.120:0/3384848799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f93c6d5-4d0a-4a81-9e56-931c78ec163d"}]: dispatch 2024-02-14T17:16:41.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:41 smithi082 ceph-mon[25796]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f93c6d5-4d0a-4a81-9e56-931c78ec163d"}]: dispatch 2024-02-14T17:16:41.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:41 smithi082 ceph-mon[25796]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f93c6d5-4d0a-4a81-9e56-931c78ec163d"}]': finished 2024-02-14T17:16:41.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:41 smithi082 ceph-mon[25796]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T17:16:41.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:41 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:42.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:42 smithi082 ceph-mon[25796]: from='client.? 172.21.15.120:0/4220809522' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:16:42.277 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:42 smithi091 ceph-mon[30290]: from='client.? 172.21.15.120:0/4220809522' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:16:42.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:42 smithi120 ceph-mon[30899]: from='client.? 172.21.15.120:0/4220809522' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:16:43.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:43 smithi120 ceph-mon[30899]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:43.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:43 smithi091 ceph-mon[30290]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:43.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:43 smithi082 ceph-mon[25796]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:45.287 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:45 smithi120 ceph-mon[30899]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:45.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:45 smithi091 ceph-mon[30290]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:45.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:45 smithi082 ceph-mon[25796]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:47.201 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:47 smithi120 ceph-mon[30899]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:47.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:47 smithi091 ceph-mon[30290]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:47.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:47 smithi082 ceph-mon[25796]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:48.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T17:16:48.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:48 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:48.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T17:16:48.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:48 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:48.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T17:16:48.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:48 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:49.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:49 smithi120 ceph-mon[30899]: Deploying daemon osd.2 on smithi120 2024-02-14T17:16:49.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:49 smithi120 ceph-mon[30899]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:49.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:49 smithi091 ceph-mon[30290]: Deploying daemon osd.2 on smithi120 2024-02-14T17:16:49.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:49 smithi091 ceph-mon[30290]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:49.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:49 smithi082 ceph-mon[25796]: Deploying daemon osd.2 on smithi120 2024-02-14T17:16:49.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:49 smithi082 ceph-mon[25796]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:51.270 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:51 smithi120 ceph-mon[30899]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:51.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:51 smithi091 ceph-mon[30290]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:51.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:51 smithi082 ceph-mon[25796]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:53.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:53 smithi120 ceph-mon[30899]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:53.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:53 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:53.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:53 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:53.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:53 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:53.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:53 smithi091 ceph-mon[30290]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:53.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:53 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:53.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:53 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:53.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:53 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:53.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:53 smithi082 ceph-mon[25796]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:53.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:53 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:53.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:53 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:53.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:53 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:55.214 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:55 smithi120 ceph-mon[30899]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:55.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:55 smithi091 ceph-mon[30290]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:55.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:55 smithi082 ceph-mon[25796]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:56.199 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 2 on host 'smithi120' 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='osd.2 [v2:172.21.15.120:6800/3641781311,v1:172.21.15.120:6801/3641781311]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.799 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:56 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:56.830 DEBUG:teuthology.orchestra.run.smithi120:osd.2> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.2.service 2024-02-14T17:16:56.832 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-02-14T17:16:56.832 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd stat -f json 2024-02-14T17:16:56.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='osd.2 [v2:172.21.15.120:6800/3641781311,v1:172.21.15.120:6801/3641781311]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:16:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:16:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:56 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:56.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:16:56.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:16:56.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:56.863 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='osd.2 [v2:172.21.15.120:6800/3641781311,v1:172.21.15.120:6801/3641781311]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:16:56.863 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:16:56.863 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.863 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:16:56.863 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:56 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:16:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:16:57.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:57 smithi120 ceph-mon[30899]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T17:16:57.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:57 smithi120 ceph-mon[30899]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T17:16:57.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:57 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:57.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:57 smithi120 ceph-mon[30899]: from='osd.2 [v2:172.21.15.120:6800/3641781311,v1:172.21.15.120:6801/3641781311]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-02-14T17:16:57.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:57 smithi120 ceph-mon[30899]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-02-14T17:16:57.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:57 smithi082 ceph-mon[25796]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T17:16:57.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:57 smithi082 ceph-mon[25796]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T17:16:57.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:57 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:57.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:57 smithi082 ceph-mon[25796]: from='osd.2 [v2:172.21.15.120:6800/3641781311,v1:172.21.15.120:6801/3641781311]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-02-14T17:16:57.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:57 smithi082 ceph-mon[25796]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-02-14T17:16:58.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:57 smithi091 ceph-mon[30290]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T17:16:58.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:57 smithi091 ceph-mon[30290]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T17:16:58.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:57 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:58.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:57 smithi091 ceph-mon[30290]: from='osd.2 [v2:172.21.15.120:6800/3641781311,v1:172.21.15.120:6801/3641781311]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-02-14T17:16:58.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:57 smithi091 ceph-mon[30290]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-02-14T17:16:58.154 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:16:58 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2[36350]: 2024-02-14T17:16:58.054+0000 7efea420e700 -1 osd.2 0 waiting for initial osdmap 2024-02-14T17:16:58.154 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:16:58 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2[36350]: 2024-02-14T17:16:58.056+0000 7efe9bff9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:16:58.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:58 smithi082 ceph-mon[25796]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-02-14T17:16:58.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:58 smithi082 ceph-mon[25796]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T17:16:58.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:58 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:58.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:58 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:58.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:58 smithi082 ceph-mon[25796]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:58.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:58 smithi082 ceph-mon[25796]: from='osd.2 ' entity='osd.2' 2024-02-14T17:16:59.035 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:16:59.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:58 smithi120 ceph-mon[30899]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-02-14T17:16:59.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:58 smithi120 ceph-mon[30899]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T17:16:59.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:58 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:59.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:58 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:59.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:58 smithi120 ceph-mon[30899]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:59.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:58 smithi120 ceph-mon[30899]: from='osd.2 ' entity='osd.2' 2024-02-14T17:16:59.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:58 smithi091 ceph-mon[30290]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-02-14T17:16:59.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:58 smithi091 ceph-mon[30290]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T17:16:59.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:58 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:59.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:58 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:59.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:58 smithi091 ceph-mon[30290]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:16:59.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:58 smithi091 ceph-mon[30290]: from='osd.2 ' entity='osd.2' 2024-02-14T17:16:59.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:59 smithi082 ceph-mon[25796]: purged_snaps scrub starts 2024-02-14T17:16:59.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:59 smithi082 ceph-mon[25796]: purged_snaps scrub ok 2024-02-14T17:16:59.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:59 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:59.698 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:59 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2384652912' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:16:59.698 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:59 smithi082 ceph-mon[25796]: osd.2 [v2:172.21.15.120:6800/3641781311,v1:172.21.15.120:6801/3641781311] boot 2024-02-14T17:16:59.698 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:59 smithi082 ceph-mon[25796]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T17:16:59.698 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:16:59 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:59.724 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"num_osds":3,"num_up_osds":2,"osd_up_since":1707930994,"num_in_osds":3,"osd_in_since":1707931000,"num_remapped_pgs":0} 2024-02-14T17:16:59.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:59 smithi120 ceph-mon[30899]: purged_snaps scrub starts 2024-02-14T17:16:59.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:59 smithi120 ceph-mon[30899]: purged_snaps scrub ok 2024-02-14T17:16:59.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:59 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:16:59.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:59 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2384652912' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:16:59.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:59 smithi120 ceph-mon[30899]: osd.2 [v2:172.21.15.120:6800/3641781311,v1:172.21.15.120:6801/3641781311] boot 2024-02-14T17:16:59.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:59 smithi120 ceph-mon[30899]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T17:16:59.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:16:59 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:17:00.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:59 smithi091 ceph-mon[30290]: purged_snaps scrub starts 2024-02-14T17:17:00.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:59 smithi091 ceph-mon[30290]: purged_snaps scrub ok 2024-02-14T17:17:00.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:59 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:17:00.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:59 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2384652912' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:17:00.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:59 smithi091 ceph-mon[30290]: osd.2 [v2:172.21.15.120:6800/3641781311,v1:172.21.15.120:6801/3641781311] boot 2024-02-14T17:17:00.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:59 smithi091 ceph-mon[30290]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T17:17:00.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:16:59 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:17:00.725 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd stat -f json 2024-02-14T17:17:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:01.081 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: Detected new or changed devices on smithi120 2024-02-14T17:17:01.081 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.081 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.081 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:17:01.082 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: Adjusting osd_memory_target on smithi120 to 17088M 2024-02-14T17:17:01.082 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.082 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:01.082 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:01.082 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.082 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: pgmap v83: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:01.082 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:00 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:17:01.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: Detected new or changed devices on smithi120 2024-02-14T17:17:01.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:17:01.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: Adjusting osd_memory_target on smithi120 to 17088M 2024-02-14T17:17:01.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:01.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:01.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: pgmap v83: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:01.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:00 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:17:01.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: Detected new or changed devices on smithi120 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: Adjusting osd_memory_target on smithi120 to 17088M 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: pgmap v83: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:01.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:00 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:17:02.095 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:01 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T17:17:02.095 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:01 smithi082 ceph-mon[25796]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T17:17:02.095 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:01 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:17:02.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:01 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T17:17:02.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:01 smithi120 ceph-mon[30899]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T17:17:02.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:01 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:17:02.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:01 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T17:17:02.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:01 smithi091 ceph-mon[30290]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T17:17:02.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:01 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:17:02.388 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:17:02 smithi082 sudo[49738]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-14T17:17:02.388 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:17:02 smithi082 sudo[49738]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:02.388 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:17:02 smithi082 sudo[49738]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:02.388 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:17:02 smithi082 sudo[49738]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:02.732 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:17:02 smithi082 sudo[49761]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-14T17:17:02.732 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:17:02 smithi082 sudo[49761]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:02.732 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:17:02 smithi082 sudo[49761]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:02.732 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:17:02 smithi082 sudo[49761]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:02.835 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:17:03.031 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:02 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:17:03.032 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:02 smithi091 ceph-mon[30290]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T17:17:03.032 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:02 smithi091 ceph-mon[30290]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:03.032 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:02 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:03.032 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:02 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/1276203513' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:17:03.032 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:02 smithi091 ceph-mon[30290]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T17:17:03.032 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:17:02 smithi091 sudo[39985]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-14T17:17:03.032 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:17:02 smithi091 sudo[39985]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:03.032 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:17:02 smithi091 sudo[39985]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:03.032 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:17:02 smithi091 sudo[39985]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:03.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:02 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:17:03.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:02 smithi082 ceph-mon[25796]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T17:17:03.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:02 smithi082 ceph-mon[25796]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:03.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:02 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:03.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:02 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1276203513' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:17:03.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:02 smithi082 ceph-mon[25796]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T17:17:03.297 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:02 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:17:03.297 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:02 smithi120 ceph-mon[30899]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T17:17:03.297 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:02 smithi120 ceph-mon[30899]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:03.297 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:02 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:03.297 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:02 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/1276203513' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:17:03.297 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:02 smithi120 ceph-mon[30899]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T17:17:03.353 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:17:03 smithi091 sudo[39988]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-14T17:17:03.353 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:17:03 smithi091 sudo[39988]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:03.354 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:17:03 smithi091 sudo[39988]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:03.354 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:17:03 smithi091 sudo[39988]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":20,"num_osds":3,"num_up_osds":3,"osd_up_since":1707931019,"num_in_osds":3,"osd_in_since":1707931000,"num_remapped_pgs":0} 2024-02-14T17:17:03.437 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T17:17:03.549 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:17:03 smithi120 sudo[41160]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-14T17:17:03.549 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:17:03 smithi120 sudo[41160]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:03.549 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:17:03 smithi120 sudo[41160]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:03.550 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:17:03 smithi120 sudo[41160]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:03.685 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:03.805 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:17:03 smithi120 sudo[41163]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-14T17:17:03.805 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:17:03 smithi120 sudo[41163]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:03.805 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:17:03 smithi120 sudo[41163]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:03.805 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:17:03 smithi120 sudo[41163]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:04.070 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:03 smithi082 sudo[49885]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:17:04.070 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:03 smithi082 sudo[49885]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:04.070 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:03 smithi082 sudo[49885]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:04.070 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:03 smithi082 sudo[49885]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:04.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:04 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:04.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:04 smithi091 ceph-mon[30290]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-14T17:17:04.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:04 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:04.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:04 smithi091 ceph-mon[30290]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:17:04.359 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:04.359 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 ceph-mon[30899]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-14T17:17:04.359 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:04.359 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 ceph-mon[30899]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:17:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:04 smithi082 sudo[49922]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:17:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:04 smithi082 sudo[49922]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:04 smithi082 sudo[49922]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:04 smithi082 sudo[49922]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:04 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:04 smithi082 ceph-mon[25796]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-14T17:17:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:04 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:04.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:04 smithi082 ceph-mon[25796]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:17:04.612 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 sudo[41166]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:17:04.612 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 sudo[41166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:04.613 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 sudo[41166]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:04.613 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 sudo[41166]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:05.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 sudo[41169]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:17:05.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 sudo[41169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:05.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 sudo[41169]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:05.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:04 smithi120 sudo[41169]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:05.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:04 smithi091 sudo[39991]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:17:05.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:04 smithi091 sudo[39991]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:05.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:04 smithi091 sudo[39991]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:05.133 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:04 smithi091 sudo[39991]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:05.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:05.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:17:05.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:05.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:05.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:05.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:05.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:17:05.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:05.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:05.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:05.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:17:05.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:05.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:05.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:05.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:05 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:05.499 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:17:05.499 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"39c4996e-cb5c-11ee-95b9-87774f69a715","created":"2024-02-14T17:13:27.877535+0000","modified":"2024-02-14T17:17:02.900895+0000","last_up_change":"2024-02-14T17:16:59.056756+0000","last_in_change":"2024-02-14T17:16:40.489675+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T17:17:00.261086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"94a14f4e-e88f-4208-87ed-78b6945d4d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6803","nonce":1287971684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6805","nonce":1287971684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6809","nonce":1287971684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6807","nonce":1287971684}]},"public_addr":"172.21.15.82:6803/1287971684","cluster_addr":"172.21.15.82:6805/1287971684","heartbeat_back_addr":"172.21.15.82:6809/1287971684","heartbeat_front_addr":"172.21.15.82:6807/1287971684","state":["exists","up"]},{"osd":1,"uuid":"5b3cadcf-e50d-453b-8b69-ed6fd1218d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6801","nonce":881479687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6803","nonce":881479687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6807","nonce":881479687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6805","nonce":881479687}]},"public_addr":"172.21.15.91:6801/881479687","cluster_addr":"172.21.15.91:6803/881479687","heartbeat_back_addr":"172.21.15.91:6807/881479687","heartbeat_front_addr":"172.21.15.91:6805/881479687","state":["exists","up"]},{"osd":2,"uuid":"1f93c6d5-4d0a-4a81-9e56-931c78ec163d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6801","nonce":3641781311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6803","nonce":3641781311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6807","nonce":3641781311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6805","nonce":3641781311}]},"public_addr":"172.21.15.120:6801/3641781311","cluster_addr":"172.21.15.120:6803/3641781311","heartbeat_back_addr":"172.21.15.120:6807/3641781311","heartbeat_front_addr":"172.21.15.120:6805/3641781311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:16:08.776856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:16:32.987143+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:16:56.959231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2325338858":"2024-02-15T17:14:29.932234+0000","172.21.15.82:6801/1627407533":"2024-02-15T17:14:29.932234+0000","172.21.15.82:6800/1627407533":"2024-02-15T17:14:29.932234+0000","172.21.15.82:0/1060846884":"2024-02-15T17:14:29.932234+0000","172.21.15.82:0/4271096371":"2024-02-15T17:14:03.178049+0000","172.21.15.82:6801/662258082":"2024-02-15T17:14:03.178049+0000","172.21.15.82:0/4115954589":"2024-02-15T17:14:03.178049+0000","172.21.15.82:0/2740402552":"2024-02-15T17:14:29.932234+0000","172.21.15.82:0/3077878380":"2024-02-15T17:14:03.178049+0000","172.21.15.82:6800/662258082":"2024-02-15T17:14:03.178049+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T17:17:05.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:05.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:17:05.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:05.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:05.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:05.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:05 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:05.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 sudo[39994]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 sudo[39994]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 sudo[39994]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 sudo[39994]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:17:05.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:05.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:05.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:05.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:17:05.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:05.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:05.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:05.605 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:05 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:06.107 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T17:17:00.261086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-14T17:17:06.107 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-14T17:17:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:06.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:06 smithi082 ceph-mon[25796]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:17:06.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:06 smithi082 ceph-mon[25796]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:17:06.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:06 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:06.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:06 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1998009356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:17:06.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:06 smithi120 ceph-mon[30899]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:17:06.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:06 smithi120 ceph-mon[30899]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:17:06.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:06 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:06.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:06 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/1998009356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:17:06.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:06 smithi091 ceph-mon[30290]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:17:06.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:06 smithi091 ceph-mon[30290]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:17:06.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:06 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:06.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:06 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/1998009356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:17:07.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:07 smithi082 ceph-mon[25796]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:07.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:07 smithi120 ceph-mon[30899]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:07.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:07 smithi091 ceph-mon[30290]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:08.170 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-02-14T17:17:08.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:08 smithi082 ceph-mon[25796]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:08.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:08 smithi120 ceph-mon[30899]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:08.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:08 smithi091 ceph-mon[30290]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:08.835 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi082 2024-02-14T17:17:08.835 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi082=foo.a' 2024-02-14T17:17:09.081 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:17:09.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:09 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1118776245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T17:17:09.515 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:09 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/1118776245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T17:17:09.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:09 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/1118776245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T17:17:10.268 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:10 smithi120 ceph-mon[30899]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:10.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:10 smithi082 ceph-mon[25796]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:10.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:10 smithi091 ceph-mon[30290]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled rgw.foo.a update... 2024-02-14T17:17:11.701 DEBUG:teuthology.orchestra.run.smithi082:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@rgw.foo.a.service 2024-02-14T17:17:11.704 INFO:tasks.cephadm:Adding prometheus.a on smithi091 2024-02-14T17:17:11.704 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch apply prometheus '1;smithi091=a' 2024-02-14T17:17:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:17:12.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi082=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: Saving service rgw.foo.a spec with placement smithi082=foo.a;count:1 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:12.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:12 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi082=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: Saving service rgw.foo.a spec with placement smithi082=foo.a;count:1 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:12.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:12 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi082=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: Saving service rgw.foo.a spec with placement smithi082=foo.a;count:1 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-14T17:17:12.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:12.401 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:12 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:13.201 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:13 smithi120 ceph-mon[30899]: Deploying daemon rgw.foo.a on smithi082 2024-02-14T17:17:13.201 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:13 smithi120 ceph-mon[30899]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:13.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:13 smithi091 ceph-mon[30290]: Deploying daemon rgw.foo.a on smithi082 2024-02-14T17:17:13.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:13 smithi091 ceph-mon[30290]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:13.371 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:13 smithi082 ceph-mon[25796]: Deploying daemon rgw.foo.a on smithi082 2024-02-14T17:17:13.371 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:13 smithi082 ceph-mon[25796]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:13.652 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled prometheus update... 2024-02-14T17:17:14.232 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:17:14 smithi082 systemd[1]: Starting Ceph rgw.foo.a for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:17:14.255 DEBUG:teuthology.orchestra.run.smithi091:prometheus.a> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@prometheus.a.service 2024-02-14T17:17:14.258 INFO:tasks.cephadm:Adding node-exporter.a on smithi082 2024-02-14T17:17:14.258 INFO:tasks.cephadm:Adding node-exporter.b on smithi091 2024-02-14T17:17:14.258 INFO:tasks.cephadm:Adding node-exporter.c on smithi120 2024-02-14T17:17:14.259 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch apply node-exporter '3;smithi082=a;smithi091=b;smithi120=c' 2024-02-14T17:17:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:17:14.650 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:17:14 smithi082 podman[50561]: 2024-02-14 17:17:14.223499984 +0000 UTC m=+0.017665588 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:17:14.651 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:17:14 smithi082 podman[50561]: 2024-02-14 17:17:14.32297512 +0000 UTC m=+0.117140717 container create 52f0cd3e4f0ddfebeacb0b3efb8dfd71d409001890571fdf472289194266949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-14T17:17:14.651 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:17:14 smithi082 podman[50561]: 2024-02-14 17:17:14.542039426 +0000 UTC m=+0.336205032 container init 52f0cd3e4f0ddfebeacb0b3efb8dfd71d409001890571fdf472289194266949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-14T17:17:14.651 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:17:14 smithi082 podman[50561]: 2024-02-14 17:17:14.547193007 +0000 UTC m=+0.341358606 container start 52f0cd3e4f0ddfebeacb0b3efb8dfd71d409001890571fdf472289194266949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-14T17:17:14.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:14 smithi120 ceph-mon[30899]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi091=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:14.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:14 smithi120 ceph-mon[30899]: Saving service prometheus spec with placement smithi091=a;count:1 2024-02-14T17:17:14.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:14 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:14.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:14 smithi120 ceph-mon[30899]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:14.981 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:17:14 smithi082 bash[50561]: 52f0cd3e4f0ddfebeacb0b3efb8dfd71d409001890571fdf472289194266949c 2024-02-14T17:17:14.981 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:17:14 smithi082 systemd[1]: Started Ceph rgw.foo.a for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:17:14.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:14 smithi082 ceph-mon[25796]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi091=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:14.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:14 smithi082 ceph-mon[25796]: Saving service prometheus spec with placement smithi091=a;count:1 2024-02-14T17:17:14.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:14 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:14.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:14 smithi082 ceph-mon[25796]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:15.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:14 smithi091 ceph-mon[30290]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi091=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:15.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:14 smithi091 ceph-mon[30290]: Saving service prometheus spec with placement smithi091=a;count:1 2024-02-14T17:17:15.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:14 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:15.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:14 smithi091 ceph-mon[30290]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:15.916 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:15.916 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:15.916 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:15.916 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: Saving service rgw.foo.a spec with placement smithi082=foo.a;count:1 2024-02-14T17:17:15.916 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:15.916 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:15.917 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: osdmap e22: 3 total, 3 up, 3 in 2024-02-14T17:17:15.917 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/57764328' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:17:15.917 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:15.917 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:15 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:16.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: Saving service rgw.foo.a spec with placement smithi082=foo.a;count:1 2024-02-14T17:17:16.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: osdmap e22: 3 total, 3 up, 3 in 2024-02-14T17:17:16.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/57764328' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:17:16.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:15 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:16.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: Saving service rgw.foo.a spec with placement smithi082=foo.a;count:1 2024-02-14T17:17:16.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: osdmap e22: 3 total, 3 up, 3 in 2024-02-14T17:17:16.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/57764328' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:17:16.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:16.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:15 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:16.238 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled node-exporter update... 2024-02-14T17:17:16.820 DEBUG:teuthology.orchestra.run.smithi082:node-exporter.a> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@node-exporter.a.service 2024-02-14T17:17:16.822 DEBUG:teuthology.orchestra.run.smithi091:node-exporter.b> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@node-exporter.b.service 2024-02-14T17:17:16.825 DEBUG:teuthology.orchestra.run.smithi120:node-exporter.c> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@node-exporter.c.service 2024-02-14T17:17:16.827 INFO:tasks.cephadm:Adding alertmanager.a on smithi082 2024-02-14T17:17:16.827 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch apply alertmanager '1;smithi082=a' 2024-02-14T17:17:17.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:16 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/57764328' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-14T17:17:17.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:16 smithi120 ceph-mon[30899]: osdmap e23: 3 total, 3 up, 3 in 2024-02-14T17:17:17.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:16 smithi120 ceph-mon[30899]: pgmap v96: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:17.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:16 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:17.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:16 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:17.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:16 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:17.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:16 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:17.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:16 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:17.099 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:17:17.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:16 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/57764328' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-14T17:17:17.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:16 smithi091 ceph-mon[30290]: osdmap e23: 3 total, 3 up, 3 in 2024-02-14T17:17:17.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:16 smithi091 ceph-mon[30290]: pgmap v96: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:17.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:16 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:17.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:16 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:17.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:16 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:17.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:16 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:17.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:16 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:17.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:16 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/57764328' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-14T17:17:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:16 smithi082 ceph-mon[25796]: osdmap e23: 3 total, 3 up, 3 in 2024-02-14T17:17:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:16 smithi082 ceph-mon[25796]: pgmap v96: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:16 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:16 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:16 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:16 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:16 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:18.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:17 smithi082 ceph-mon[25796]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi082=a;smithi091=b;smithi120=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:17 smithi082 ceph-mon[25796]: Saving service node-exporter spec with placement smithi082=a;smithi091=b;smithi120=c;count:3 2024-02-14T17:17:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:17 smithi082 ceph-mon[25796]: Checking dashboard <-> RGW credentials 2024-02-14T17:17:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:17 smithi082 ceph-mon[25796]: osdmap e24: 3 total, 3 up, 3 in 2024-02-14T17:17:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:17 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:17:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:17 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:17:18.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:17 smithi120 ceph-mon[30899]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi082=a;smithi091=b;smithi120=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:18.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:17 smithi120 ceph-mon[30899]: Saving service node-exporter spec with placement smithi082=a;smithi091=b;smithi120=c;count:3 2024-02-14T17:17:18.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:17 smithi120 ceph-mon[30899]: Checking dashboard <-> RGW credentials 2024-02-14T17:17:18.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:17 smithi120 ceph-mon[30899]: osdmap e24: 3 total, 3 up, 3 in 2024-02-14T17:17:18.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:17 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:17:18.267 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:17 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:17:18.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:17 smithi091 ceph-mon[30290]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi082=a;smithi091=b;smithi120=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:18.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:17 smithi091 ceph-mon[30290]: Saving service node-exporter spec with placement smithi082=a;smithi091=b;smithi120=c;count:3 2024-02-14T17:17:18.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:17 smithi091 ceph-mon[30290]: Checking dashboard <-> RGW credentials 2024-02-14T17:17:18.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:17 smithi091 ceph-mon[30290]: osdmap e24: 3 total, 3 up, 3 in 2024-02-14T17:17:18.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:17 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:17:18.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:17 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:17:18.946 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled alertmanager update... 2024-02-14T17:17:19.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:18 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:17:19.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:18 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:17:19.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:18 smithi082 ceph-mon[25796]: osdmap e25: 3 total, 3 up, 3 in 2024-02-14T17:17:19.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:18 smithi082 ceph-mon[25796]: pgmap v99: 65 pgs: 10 creating+peering, 46 unknown, 9 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-14T17:17:19.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:18 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:17:19.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:18 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:17:19.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:18 smithi120 ceph-mon[30899]: osdmap e25: 3 total, 3 up, 3 in 2024-02-14T17:17:19.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:18 smithi120 ceph-mon[30899]: pgmap v99: 65 pgs: 10 creating+peering, 46 unknown, 9 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-14T17:17:19.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:18 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:17:19.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:18 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:17:19.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:18 smithi091 ceph-mon[30290]: osdmap e25: 3 total, 3 up, 3 in 2024-02-14T17:17:19.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:18 smithi091 ceph-mon[30290]: pgmap v99: 65 pgs: 10 creating+peering, 46 unknown, 9 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-14T17:17:19.611 DEBUG:teuthology.orchestra.run.smithi082:alertmanager.a> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@alertmanager.a.service 2024-02-14T17:17:19.614 INFO:tasks.cephadm:Adding grafana.a on smithi091 2024-02-14T17:17:19.614 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph orch apply grafana '1;smithi091=a' 2024-02-14T17:17:19.849 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:17:20.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:19 smithi082 ceph-mon[25796]: osdmap e26: 3 total, 3 up, 3 in 2024-02-14T17:17:20.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:19 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:17:20.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:19 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:17:20.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:19 smithi082 ceph-mon[25796]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi082=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:20.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:19 smithi082 ceph-mon[25796]: Saving service alertmanager spec with placement smithi082=a;count:1 2024-02-14T17:17:20.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:19 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:20.235 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:19 smithi120 ceph-mon[30899]: osdmap e26: 3 total, 3 up, 3 in 2024-02-14T17:17:20.235 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:19 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:17:20.235 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:19 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:17:20.235 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:19 smithi120 ceph-mon[30899]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi082=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:20.235 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:19 smithi120 ceph-mon[30899]: Saving service alertmanager spec with placement smithi082=a;count:1 2024-02-14T17:17:20.235 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:19 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:20.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:19 smithi091 ceph-mon[30290]: osdmap e26: 3 total, 3 up, 3 in 2024-02-14T17:17:20.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:19 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:17:20.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:19 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:17:20.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:19 smithi091 ceph-mon[30290]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi082=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:20.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:19 smithi091 ceph-mon[30290]: Saving service alertmanager spec with placement smithi082=a;count:1 2024-02-14T17:17:20.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:19 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:21.054 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:20 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:17:21.054 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:20 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:17:21.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:20 smithi120 ceph-mon[30899]: osdmap e27: 3 total, 3 up, 3 in 2024-02-14T17:17:21.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:20 smithi120 ceph-mon[30899]: pgmap v102: 97 pgs: 22 creating+peering, 43 unknown, 32 active+clean; 578 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:17:21.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:20 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:17:21.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:20 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:17:21.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:20 smithi082 ceph-mon[25796]: osdmap e27: 3 total, 3 up, 3 in 2024-02-14T17:17:21.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:20 smithi082 ceph-mon[25796]: pgmap v102: 97 pgs: 22 creating+peering, 43 unknown, 32 active+clean; 578 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:17:21.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:20 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:17:21.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:20 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:17:21.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:20 smithi091 ceph-mon[30290]: osdmap e27: 3 total, 3 up, 3 in 2024-02-14T17:17:21.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:20 smithi091 ceph-mon[30290]: pgmap v102: 97 pgs: 22 creating+peering, 43 unknown, 32 active+clean; 578 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:17:21.580 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled grafana update... 2024-02-14T17:17:22.104 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:21 smithi120 ceph-mon[30899]: osdmap e28: 3 total, 3 up, 3 in 2024-02-14T17:17:22.104 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:21 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:17:22.104 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:21 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:17:22.104 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:21 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:22.129 DEBUG:teuthology.orchestra.run.smithi091:grafana.a> sudo journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@grafana.a.service 2024-02-14T17:17:22.132 INFO:tasks.cephadm:Setting up client nodes... 2024-02-14T17:17:22.132 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-14T17:17:22.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:21 smithi082 ceph-mon[25796]: osdmap e28: 3 total, 3 up, 3 in 2024-02-14T17:17:22.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:21 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:17:22.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:21 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:17:22.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:21 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:22.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:21 smithi091 ceph-mon[30290]: osdmap e28: 3 total, 3 up, 3 in 2024-02-14T17:17:22.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:21 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:17:22.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:21 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:17:22.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:21 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:22.375 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:23.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi091=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:23.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: Saving service grafana spec with placement smithi091=a;count:1 2024-02-14T17:17:23.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:17:23.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:17:23.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: osdmap e29: 3 total, 3 up, 3 in 2024-02-14T17:17:23.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:17:23.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:17:23.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: pgmap v105: 129 pgs: 22 creating+peering, 22 unknown, 85 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-02-14T17:17:23.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:17:23.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:17:23.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:22 smithi082 ceph-mon[25796]: osdmap e30: 3 total, 3 up, 3 in 2024-02-14T17:17:23.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi091=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:23.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: Saving service grafana spec with placement smithi091=a;count:1 2024-02-14T17:17:23.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:17:23.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:17:23.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: osdmap e29: 3 total, 3 up, 3 in 2024-02-14T17:17:23.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:17:23.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:17:23.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: pgmap v105: 129 pgs: 22 creating+peering, 22 unknown, 85 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-02-14T17:17:23.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:17:23.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:17:23.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:22 smithi120 ceph-mon[30899]: osdmap e30: 3 total, 3 up, 3 in 2024-02-14T17:17:23.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi091=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:17:23.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: Saving service grafana spec with placement smithi091=a;count:1 2024-02-14T17:17:23.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:17:23.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:17:23.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: osdmap e29: 3 total, 3 up, 3 in 2024-02-14T17:17:23.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:17:23.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:17:23.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: pgmap v105: 129 pgs: 22 creating+peering, 22 unknown, 85 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-02-14T17:17:23.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/4204311037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:17:23.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2959868064' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:17:23.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:22 smithi091 ceph-mon[30290]: osdmap e30: 3 total, 3 up, 3 in 2024-02-14T17:17:24.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:24 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:24.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:24 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:24.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:24 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:24.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:24 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:24.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:24 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:24.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:24 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:24.453 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-02-14T17:17:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQCk9cxlIme/GhAA3YiBVKgH6xzyMZ13z79qUA== 2024-02-14T17:17:24.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:24 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:24.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:24 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:24.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:24 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:25.094 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:17:25.094 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-14T17:17:25.094 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-14T17:17:25.136 DEBUG:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-14T17:17:25.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:25 smithi091 ceph-mon[30290]: Deploying daemon prometheus.a on smithi091 2024-02-14T17:17:25.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:25 smithi091 ceph-mon[30290]: pgmap v107: 129 pgs: 22 creating+peering, 22 unknown, 85 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 503 B/s wr, 3 op/s 2024-02-14T17:17:25.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:25 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2089309977' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:25.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:25 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2089309977' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:17:25.375 INFO:teuthology.orchestra.run.smithi091.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:17:25.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:25 smithi082 ceph-mon[25796]: Deploying daemon prometheus.a on smithi091 2024-02-14T17:17:25.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:25 smithi082 ceph-mon[25796]: pgmap v107: 129 pgs: 22 creating+peering, 22 unknown, 85 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 503 B/s wr, 3 op/s 2024-02-14T17:17:25.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:25 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2089309977' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:25.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:25 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2089309977' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:17:25.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:25 smithi120 ceph-mon[30899]: Deploying daemon prometheus.a on smithi091 2024-02-14T17:17:25.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:25 smithi120 ceph-mon[30899]: pgmap v107: 129 pgs: 22 creating+peering, 22 unknown, 85 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 503 B/s wr, 3 op/s 2024-02-14T17:17:25.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:25 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2089309977' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:25.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:25 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2089309977' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:17:27.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:27 smithi091 ceph-mon[30290]: pgmap v108: 129 pgs: 10 creating+peering, 119 active+clean; 581 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 50 KiB/s rd, 4.2 KiB/s wr, 121 op/s 2024-02-14T17:17:27.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:27 smithi082 ceph-mon[25796]: pgmap v108: 129 pgs: 10 creating+peering, 119 active+clean; 581 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 50 KiB/s rd, 4.2 KiB/s wr, 121 op/s 2024-02-14T17:17:27.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:27 smithi120 ceph-mon[30899]: pgmap v108: 129 pgs: 10 creating+peering, 119 active+clean; 581 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 50 KiB/s rd, 4.2 KiB/s wr, 121 op/s 2024-02-14T17:17:28.335 INFO:teuthology.orchestra.run.smithi091.stdout:[client.1] 2024-02-14T17:17:28.335 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCo9cxl09OwExAAONnn0M0kVSd4RUFCtpv3QQ== 2024-02-14T17:17:29.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:29 smithi082 ceph-mon[25796]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 65 KiB/s rd, 5.0 KiB/s wr, 158 op/s 2024-02-14T17:17:29.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:29 smithi082 ceph-mon[25796]: from='client.? 172.21.15.91:0/254953809' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:29.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:29 smithi082 ceph-mon[25796]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:29.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:29 smithi082 ceph-mon[25796]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:17:29.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:29 smithi120 ceph-mon[30899]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 65 KiB/s rd, 5.0 KiB/s wr, 158 op/s 2024-02-14T17:17:29.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:29 smithi120 ceph-mon[30899]: from='client.? 172.21.15.91:0/254953809' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:29.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:29 smithi120 ceph-mon[30899]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:29.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:29 smithi120 ceph-mon[30899]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:17:29.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:29 smithi091 ceph-mon[30290]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 65 KiB/s rd, 5.0 KiB/s wr, 158 op/s 2024-02-14T17:17:29.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:29 smithi091 ceph-mon[30290]: from='client.? 172.21.15.91:0/254953809' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:29.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:29 smithi091 ceph-mon[30290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:29.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:29 smithi091 ceph-mon[30290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:17:30.682 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:17:30.682 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-14T17:17:30.682 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-14T17:17:30.725 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-14T17:17:30.970 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T17:17:31.423 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:31 smithi120 ceph-mon[30899]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 138 op/s 2024-02-14T17:17:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:31 smithi082 ceph-mon[25796]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 138 op/s 2024-02-14T17:17:31.562 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:31 smithi091 ceph-mon[30290]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 138 op/s 2024-02-14T17:17:32.816 INFO:teuthology.orchestra.run.smithi120.stdout:[client.2] 2024-02-14T17:17:32.816 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCs9cxlNZxjMBAAPvmzRnpi+YofZ3Znj+OMEg== 2024-02-14T17:17:33.317 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:33 smithi091 ceph-mon[30290]: pgmap v111: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-02-14T17:17:33.317 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:33 smithi091 ceph-mon[30290]: from='client.? 172.21.15.120:0/1679369436' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:33.317 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:33 smithi091 ceph-mon[30290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:33.317 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:33 smithi091 ceph-mon[30290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:17:33.449 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:33 smithi120 ceph-mon[30899]: pgmap v111: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-02-14T17:17:33.449 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:33 smithi120 ceph-mon[30899]: from='client.? 172.21.15.120:0/1679369436' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:33.449 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:33 smithi120 ceph-mon[30899]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:33.449 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:33 smithi120 ceph-mon[30899]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:17:33.478 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:17:33.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-02-14T17:17:33.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-14T17:17:33.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:33 smithi082 ceph-mon[25796]: pgmap v111: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-02-14T17:17:33.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:33 smithi082 ceph-mon[25796]: from='client.? 172.21.15.120:0/1679369436' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:33.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:33 smithi082 ceph-mon[25796]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:17:33.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:33 smithi082 ceph-mon[25796]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:17:33.521 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-14T17:17:33.521 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-14T17:17:33.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph mgr dump --format=json 2024-02-14T17:17:33.768 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:34.604 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:34 smithi091 systemd[1]: Starting Ceph prometheus.a for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:17:35.103 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:34 smithi091 podman[40899]: 2024-02-14 17:17:34.63844231 +0000 UTC m=+0.098515741 container create 829071073b8136ad97c67af30409e518c8d429a3277d9a59c873a13b5ef9b851 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:17:35.103 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:34 smithi091 podman[40899]: 2024-02-14 17:17:34.558465716 +0000 UTC m=+0.018539154 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-14T17:17:35.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:35 smithi082 ceph-mon[25796]: pgmap v112: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.1 KiB/s wr, 100 op/s 2024-02-14T17:17:35.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:35 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:35.491 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-mon[30290]: pgmap v112: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.1 KiB/s wr, 100 op/s 2024-02-14T17:17:35.491 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:35.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:35 smithi120 ceph-mon[30899]: pgmap v112: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.1 KiB/s wr, 100 op/s 2024-02-14T17:17:35.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:35 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:35.621 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:17:35.854 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 podman[40899]: 2024-02-14 17:17:35.490997783 +0000 UTC m=+0.951071212 container init 829071073b8136ad97c67af30409e518c8d429a3277d9a59c873a13b5ef9b851 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:17:35.854 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 podman[40899]: 2024-02-14 17:17:35.497110301 +0000 UTC m=+0.957183732 container start 829071073b8136ad97c67af30409e518c8d429a3277d9a59c873a13b5ef9b851 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:17:35.854 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.523Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-14T17:17:35.854 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.523Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-14T17:17:35.854 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.523Z caller=main.go:570 level=info host_details="(Linux 5.14.0-420.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 9 18:10:28 UTC 2024 x86_64 smithi091 (none))" 2024-02-14T17:17:35.854 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.523Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-14T17:17:35.854 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.523Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-14T17:17:35.854 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.524Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.525Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.527Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.527Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.528Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.528Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.792µs 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.528Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.528Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.528Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=31.402µs wal_replay_duration=263.715µs wbl_replay_duration=124ns total_replay_duration=320.472µs 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.529Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.529Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.529Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-14T17:17:35.855 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.547Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=18.347468ms db_storage=892ns remote_storage=1.739µs web_handler=200ns query_engine=614ns scrape=460.561µs scrape_sd=70.956µs notify=800ns notify_sd=1.26µs rules=17.613226ms tracing=6.603µs 2024-02-14T17:17:35.856 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.547Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-14T17:17:35.856 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:17:35.547Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-14T17:17:35.856 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 bash[40899]: 829071073b8136ad97c67af30409e518c8d429a3277d9a59c873a13b5ef9b851 2024-02-14T17:17:35.856 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:17:35 smithi091 systemd[1]: Started Ceph prometheus.a for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:17:36.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:36 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1907015834' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T17:17:36.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:36.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:36.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:36.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:36 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-14T17:17:36.237 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1191253866},{"type":"v1","addr":"172.21.15.82:6801","nonce":1191253866}]},"active_addr":"172.21.15.82:6801/1191253866","active_change":"2024-02-14T17:14:29.932361+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.82:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2361383234}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1986774944}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3871990523}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3980237879}]}]} 2024-02-14T17:17:36.243 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-14T17:17:36.243 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-14T17:17:36.244 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T17:17:36.476 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:36.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:36 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/1907015834' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T17:17:36.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:36.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:36.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:36.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:36 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-14T17:17:36.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:36 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/1907015834' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T17:17:36.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:36.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:36.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' 2024-02-14T17:17:36.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:36 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-14T17:17:36.882 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:36 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ignoring --setuser ceph since I am not root 2024-02-14T17:17:36.882 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:36 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ignoring --setgroup ceph since I am not root 2024-02-14T17:17:37.055 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:36 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: ignoring --setuser ceph since I am not root 2024-02-14T17:17:37.055 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:36 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: ignoring --setgroup ceph since I am not root 2024-02-14T17:17:37.055 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:36 smithi120 ceph-mgr[33177]: -- 172.21.15.120:0/787371352 <== mon.1 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55be637841a0 con 0x55be6375ac00 2024-02-14T17:17:37.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:36 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: ignoring --setuser ceph since I am not root 2024-02-14T17:17:37.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:36 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: ignoring --setgroup ceph since I am not root 2024-02-14T17:17:37.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:36 smithi091 ceph-mgr[31783]: -- 172.21.15.91:0/654754054 <== mon.2 v2:172.21.15.91:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555ddf63a1a0 con 0x555ddf610800 2024-02-14T17:17:37.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:37 smithi082 ceph-mon[25796]: pgmap v113: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.8 KiB/s wr, 92 op/s 2024-02-14T17:17:37.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:37 smithi082 ceph-mon[25796]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-14T17:17:37.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:37 smithi082 ceph-mon[25796]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-14T17:17:37.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:37 smithi120 ceph-mon[30899]: pgmap v113: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.8 KiB/s wr, 92 op/s 2024-02-14T17:17:37.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:37 smithi120 ceph-mon[30899]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-14T17:17:37.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:37 smithi120 ceph-mon[30899]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-14T17:17:37.555 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:37 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:37.240+0000 7fedabbf8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:17:37.556 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:37 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:37.331+0000 7fedabbf8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:17:37.593 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:37 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:37.256+0000 7fe6f52c4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:17:37.594 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:37 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:37.351+0000 7fe6f52c4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:17:37.603 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:37.250+0000 7ff8257a9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:17:37.603 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:37.343+0000 7ff8257a9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:17:37.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:37 smithi091 ceph-mon[30290]: pgmap v113: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.8 KiB/s wr, 92 op/s 2024-02-14T17:17:37.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:37 smithi091 ceph-mon[30290]: from='mgr.14152 172.21.15.82:0/512743703' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-14T17:17:37.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:37 smithi091 ceph-mon[30290]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-14T17:17:38.232 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:38.008+0000 7fe6f52c4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:17:38.232 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:38.104+0000 7fe6f52c4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:17:38.277 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:37 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:37.955+0000 7fedabbf8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:17:38.277 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:38 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:38.045+0000 7fedabbf8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:17:38.278 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:38 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:38.183+0000 7fedabbf8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:17:38.278 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:38 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:38.276+0000 7fedabbf8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:17:38.307 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:37.977+0000 7ff8257a9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:17:38.307 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:38.070+0000 7ff8257a9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:17:38.307 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:38.210+0000 7ff8257a9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:17:38.307 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:38.305+0000 7ff8257a9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:17:38.350 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:17:38.350 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":30,"fsid":"39c4996e-cb5c-11ee-95b9-87774f69a715","created":"2024-02-14T17:13:27.877535+0000","modified":"2024-02-14T17:17:22.909813+0000","last_up_change":"2024-02-14T17:16:59.056756+0000","last_in_change":"2024-02-14T17:16:40.489675+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T17:17:00.261086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-14T17:17:14.820429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-14T17:17:16.179278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-14T17:17:18.130044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-14T17:17:20.193065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"94a14f4e-e88f-4208-87ed-78b6945d4d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6803","nonce":1287971684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6805","nonce":1287971684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6809","nonce":1287971684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6807","nonce":1287971684}]},"public_addr":"172.21.15.82:6803/1287971684","cluster_addr":"172.21.15.82:6805/1287971684","heartbeat_back_addr":"172.21.15.82:6809/1287971684","heartbeat_front_addr":"172.21.15.82:6807/1287971684","state":["exists","up"]},{"osd":1,"uuid":"5b3cadcf-e50d-453b-8b69-ed6fd1218d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6801","nonce":881479687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6803","nonce":881479687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6807","nonce":881479687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6805","nonce":881479687}]},"public_addr":"172.21.15.91:6801/881479687","cluster_addr":"172.21.15.91:6803/881479687","heartbeat_back_addr":"172.21.15.91:6807/881479687","heartbeat_front_addr":"172.21.15.91:6805/881479687","state":["exists","up"]},{"osd":2,"uuid":"1f93c6d5-4d0a-4a81-9e56-931c78ec163d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6801","nonce":3641781311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6803","nonce":3641781311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6807","nonce":3641781311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6805","nonce":3641781311}]},"public_addr":"172.21.15.120:6801/3641781311","cluster_addr":"172.21.15.120:6803/3641781311","heartbeat_back_addr":"172.21.15.120:6807/3641781311","heartbeat_front_addr":"172.21.15.120:6805/3641781311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:16:08.776856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:16:32.987143+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:16:56.959231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2325338858":"2024-02-15T17:14:29.932234+0000","172.21.15.82:6801/1627407533":"2024-02-15T17:14:29.932234+0000","172.21.15.82:6800/1627407533":"2024-02-15T17:14:29.932234+0000","172.21.15.82:0/1060846884":"2024-02-15T17:14:29.932234+0000","172.21.15.82:0/4271096371":"2024-02-15T17:14:03.178049+0000","172.21.15.82:6801/662258082":"2024-02-15T17:14:03.178049+0000","172.21.15.82:0/4115954589":"2024-02-15T17:14:03.178049+0000","172.21.15.82:0/2740402552":"2024-02-15T17:14:29.932234+0000","172.21.15.82:0/3077878380":"2024-02-15T17:14:03.178049+0000","172.21.15.82:6800/662258082":"2024-02-15T17:14:03.178049+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T17:17:38.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:38 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/1373399283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:17:38.555 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:38 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:38.357+0000 7fedabbf8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:17:38.603 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:38 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/1373399283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:17:38.604 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:38.388+0000 7ff8257a9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:17:38.649 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1373399283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:17:38.650 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:38.251+0000 7fe6f52c4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:17:38.650 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:38.347+0000 7fe6f52c4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:17:38.650 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:38.436+0000 7fe6f52c4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:17:38.900 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:38 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:38.616+0000 7fedabbf8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:17:38.900 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:38 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:38.722+0000 7fedabbf8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:17:38.900 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:38 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:38.807+0000 7fedabbf8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:17:38.900 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:38 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:38.898+0000 7fedabbf8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:17:38.916 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:38.701+0000 7fe6f52c4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:17:38.916 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:38.810+0000 7fe6f52c4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:17:38.916 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:38.897+0000 7fe6f52c4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:17:38.944 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-14T17:17:38.944 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T17:17:38.953 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:38.660+0000 7ff8257a9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:17:38.953 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:38.771+0000 7ff8257a9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:17:38.953 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:38.857+0000 7ff8257a9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:17:38.953 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:38.951+0000 7ff8257a9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:17:39.184 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:39.185 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:38 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:38.998+0000 7fe6f52c4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:17:39.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:39 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:39.070+0000 7fedabbf8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:17:39.355 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:39 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:39.125+0000 7ff8257a9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:17:39.437 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:39 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:39.183+0000 7fe6f52c4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:17:40.231 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:40 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:40.189+0000 7fe6f52c4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:17:40.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:40 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:40.014+0000 7fedabbf8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:17:40.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:40 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:40.142+0000 7fedabbf8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:17:40.353 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:40 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:40.081+0000 7ff8257a9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:17:40.353 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:40 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:40.211+0000 7ff8257a9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:17:40.605 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:40 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:40.341+0000 7fe6f52c4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:17:40.774 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:40 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:40.477+0000 7fedabbf8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:17:40.774 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:40 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:40.571+0000 7fedabbf8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:17:40.774 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:40 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:40.772+0000 7fedabbf8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:17:40.830 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:40 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:40.542+0000 7ff8257a9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:17:40.830 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:40 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:40.632+0000 7ff8257a9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:17:40.981 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:40 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:40.683+0000 7fe6f52c4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:17:40.982 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:40 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:40.776+0000 7fe6f52c4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:17:40.982 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:40 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:40.979+0000 7fe6f52c4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:17:41.052 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:17:41.052 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":30,"fsid":"39c4996e-cb5c-11ee-95b9-87774f69a715","created":"2024-02-14T17:13:27.877535+0000","modified":"2024-02-14T17:17:22.909813+0000","last_up_change":"2024-02-14T17:16:59.056756+0000","last_in_change":"2024-02-14T17:16:40.489675+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T17:17:00.261086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-14T17:17:14.820429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-14T17:17:16.179278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-14T17:17:18.130044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-14T17:17:20.193065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"94a14f4e-e88f-4208-87ed-78b6945d4d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6803","nonce":1287971684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6805","nonce":1287971684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6809","nonce":1287971684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1287971684},{"type":"v1","addr":"172.21.15.82:6807","nonce":1287971684}]},"public_addr":"172.21.15.82:6803/1287971684","cluster_addr":"172.21.15.82:6805/1287971684","heartbeat_back_addr":"172.21.15.82:6809/1287971684","heartbeat_front_addr":"172.21.15.82:6807/1287971684","state":["exists","up"]},{"osd":1,"uuid":"5b3cadcf-e50d-453b-8b69-ed6fd1218d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6801","nonce":881479687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6803","nonce":881479687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6807","nonce":881479687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":881479687},{"type":"v1","addr":"172.21.15.91:6805","nonce":881479687}]},"public_addr":"172.21.15.91:6801/881479687","cluster_addr":"172.21.15.91:6803/881479687","heartbeat_back_addr":"172.21.15.91:6807/881479687","heartbeat_front_addr":"172.21.15.91:6805/881479687","state":["exists","up"]},{"osd":2,"uuid":"1f93c6d5-4d0a-4a81-9e56-931c78ec163d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6801","nonce":3641781311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6803","nonce":3641781311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6807","nonce":3641781311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3641781311},{"type":"v1","addr":"172.21.15.120:6805","nonce":3641781311}]},"public_addr":"172.21.15.120:6801/3641781311","cluster_addr":"172.21.15.120:6803/3641781311","heartbeat_back_addr":"172.21.15.120:6807/3641781311","heartbeat_front_addr":"172.21.15.120:6805/3641781311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:16:08.776856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:16:32.987143+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:16:56.959231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2325338858":"2024-02-15T17:14:29.932234+0000","172.21.15.82:6801/1627407533":"2024-02-15T17:14:29.932234+0000","172.21.15.82:6800/1627407533":"2024-02-15T17:14:29.932234+0000","172.21.15.82:0/1060846884":"2024-02-15T17:14:29.932234+0000","172.21.15.82:0/4271096371":"2024-02-15T17:14:03.178049+0000","172.21.15.82:6801/662258082":"2024-02-15T17:14:03.178049+0000","172.21.15.82:0/4115954589":"2024-02-15T17:14:03.178049+0000","172.21.15.82:0/2740402552":"2024-02-15T17:14:29.932234+0000","172.21.15.82:0/3077878380":"2024-02-15T17:14:03.178049+0000","172.21.15.82:6800/662258082":"2024-02-15T17:14:03.178049+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T17:17:41.055 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:40 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:40.960+0000 7fedabbf8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:17:41.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:40 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:40.828+0000 7ff8257a9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:17:41.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:41 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:41.014+0000 7ff8257a9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:17:41.367 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:41 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2783385655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:17:41.368 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:41 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:41.245+0000 7fedabbf8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:17:41.419 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:41 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2783385655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:17:41.420 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:41 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:41.171+0000 7fe6f52c4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:17:41.553 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:41 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2783385655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:17:41.554 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:41 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:41.297+0000 7ff8257a9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:17:41.554 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:41 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:41.552+0000 7ff8257a9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:17:41.703 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:41 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:41.459+0000 7fe6f52c4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:17:41.731 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-14T17:17:41.731 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-14T17:17:41.732 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-14T17:17:41.734 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:41 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:41.503+0000 7fedabbf8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:17:41.734 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:41 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:41.733+0000 7fedabbf8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:17:41.853 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:41 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:41.790+0000 7ff8257a9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:17:41.981 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:41 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:41.721+0000 7fe6f52c4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:17:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:42.005 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:42.008 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:42.275 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:42 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:42.007+0000 7fe6f52c4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:17:42.805 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:42 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:42.515+0000 7fedabbf8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:17:42.853 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:42 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:42.588+0000 7ff8257a9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:17:43.155 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:42 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:42.856+0000 7fe6f52c4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:17:43.278 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:43.174+0000 7ff8257a9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:17:43.383 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:43.076+0000 7fedabbf8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:17:43.384 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:43.171+0000 7fedabbf8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:17:43.384 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:43.382+0000 7fedabbf8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:17:43.603 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:43.277+0000 7ff8257a9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:17:43.603 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:43.502+0000 7ff8257a9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:17:43.604 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: [14/Feb/2024:17:17:43] ENGINE Bus STARTING 2024-02-14T17:17:43.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-mon[30290]: Standby manager daemon z restarted 2024-02-14T17:17:43.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-mon[30290]: Standby manager daemon z started 2024-02-14T17:17:43.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T17:17:43.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:17:43.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T17:17:43.604 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:17:43.677 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:43.452+0000 7fe6f52c4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:17:43.677 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:43.567+0000 7fe6f52c4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:17:43.677 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-mon[25796]: Standby manager daemon z restarted 2024-02-14T17:17:43.677 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-mon[25796]: Standby manager daemon z started 2024-02-14T17:17:43.677 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T17:17:43.678 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:17:43.678 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T17:17:43.678 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:17:43.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-mon[30899]: Standby manager daemon z restarted 2024-02-14T17:17:43.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-mon[30899]: Standby manager daemon z started 2024-02-14T17:17:43.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T17:17:43.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:17:43.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T17:17:43.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.120:0/1200541499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:17:43.806 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: [14/Feb/2024:17:17:43] ENGINE Bus STARTING 2024-02-14T17:17:43.806 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: CherryPy Checker: 2024-02-14T17:17:43.806 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: The Application mounted at '' has an empty config. 2024-02-14T17:17:43.806 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: 2024-02-14T17:17:43.806 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: [14/Feb/2024:17:17:43] ENGINE Serving on http://:::9283 2024-02-14T17:17:43.806 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:17:43 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z[33107]: [14/Feb/2024:17:17:43] ENGINE Bus STARTED 2024-02-14T17:17:43.914 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: CherryPy Checker: 2024-02-14T17:17:43.914 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: The Application mounted at '' has an empty config. 2024-02-14T17:17:43.914 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: 2024-02-14T17:17:43.914 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: [14/Feb/2024:17:17:43] ENGINE Serving on http://:::9283 2024-02-14T17:17:43.914 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:17:43 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x[31760]: [14/Feb/2024:17:17:43] ENGINE Bus STARTED 2024-02-14T17:17:43.939 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:43.822+0000 7fe6f52c4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:17:44.232 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:43 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:17:43] ENGINE Bus STARTING 2024-02-14T17:17:44.232 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: CherryPy Checker: 2024-02-14T17:17:44.232 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: The Application mounted at '' has an empty config. 2024-02-14T17:17:44.233 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:17:44.233 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:17:44] ENGINE Serving on http://:::9283 2024-02-14T17:17:44.233 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:17:44] ENGINE Bus STARTED 2024-02-14T17:17:44.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-14T17:17:44.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: Standby manager daemon x restarted 2024-02-14T17:17:44.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: Standby manager daemon x started 2024-02-14T17:17:44.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:17:44.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:17:44.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: Active manager daemon y restarted 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: Activating manager daemon y 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: osdmap e31: 3 total, 3 up, 3 in 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: mgrmap e19: y(active, starting, since 0.00994631s), standbys: x, z 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:17:44.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:17:44.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:17:44.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:17:44.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: Manager daemon y is now available 2024-02-14T17:17:44.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:44.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:44.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:44.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:17:44.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:17:44.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:17:44.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-14T17:17:44.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: Standby manager daemon x restarted 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: Standby manager daemon x started 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: Active manager daemon y restarted 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: Activating manager daemon y 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: osdmap e31: 3 total, 3 up, 3 in 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: mgrmap e19: y(active, starting, since 0.00994631s), standbys: x, z 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:17:44.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:17:44.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T17:17:44.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:17:44.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:17:44.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:17:44.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:17:44.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:17:44.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:17:44.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: Manager daemon y is now available 2024-02-14T17:17:44.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:44.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:44.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:44.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:17:44.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:17:44.808 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:44 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: Standby manager daemon x restarted 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: Standby manager daemon x started 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.? 172.21.15.91:0/2428216211' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: Active manager daemon y restarted 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: Activating manager daemon y 2024-02-14T17:17:44.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: osdmap e31: 3 total, 3 up, 3 in 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: mgrmap e19: y(active, starting, since 0.00994631s), standbys: x, z 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:17:44.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: Manager daemon y is now available 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:17:44.856 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:44 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:17:45.232 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:44 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ::ffff:172.21.15.91 - - [14/Feb/2024:17:17:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:17:45.258 INFO:teuthology.orchestra.run.smithi082.stdout:55834574862 2024-02-14T17:17:45.258 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-14T17:17:45.703 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:45 smithi082 ceph-mon[25796]: [14/Feb/2024:17:17:44] ENGINE Bus STARTING 2024-02-14T17:17:45.703 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:45 smithi082 ceph-mon[25796]: [14/Feb/2024:17:17:44] ENGINE Serving on http://172.21.15.82:8765 2024-02-14T17:17:45.703 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:45 smithi082 ceph-mon[25796]: [14/Feb/2024:17:17:44] ENGINE Serving on https://172.21.15.82:7150 2024-02-14T17:17:45.703 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:45 smithi082 ceph-mon[25796]: [14/Feb/2024:17:17:44] ENGINE Bus STARTED 2024-02-14T17:17:45.703 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:45 smithi082 ceph-mon[25796]: mgrmap e20: y(active, since 1.01618s), standbys: x, z 2024-02-14T17:17:45.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:45 smithi120 ceph-mon[30899]: [14/Feb/2024:17:17:44] ENGINE Bus STARTING 2024-02-14T17:17:45.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:45 smithi120 ceph-mon[30899]: [14/Feb/2024:17:17:44] ENGINE Serving on http://172.21.15.82:8765 2024-02-14T17:17:45.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:45 smithi120 ceph-mon[30899]: [14/Feb/2024:17:17:44] ENGINE Serving on https://172.21.15.82:7150 2024-02-14T17:17:45.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:45 smithi120 ceph-mon[30899]: [14/Feb/2024:17:17:44] ENGINE Bus STARTED 2024-02-14T17:17:45.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:45 smithi120 ceph-mon[30899]: mgrmap e20: y(active, since 1.01618s), standbys: x, z 2024-02-14T17:17:45.820 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:45 smithi091 ceph-mon[30290]: [14/Feb/2024:17:17:44] ENGINE Bus STARTING 2024-02-14T17:17:45.820 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:45 smithi091 ceph-mon[30290]: [14/Feb/2024:17:17:44] ENGINE Serving on http://172.21.15.82:8765 2024-02-14T17:17:45.820 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:45 smithi091 ceph-mon[30290]: [14/Feb/2024:17:17:44] ENGINE Serving on https://172.21.15.82:7150 2024-02-14T17:17:45.820 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:45 smithi091 ceph-mon[30290]: [14/Feb/2024:17:17:44] ENGINE Bus STARTED 2024-02-14T17:17:45.820 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:45 smithi091 ceph-mon[30290]: mgrmap e20: y(active, since 1.01618s), standbys: x, z 2024-02-14T17:17:46.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:46 smithi082 ceph-mon[25796]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:46.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:46 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:46.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:46 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:46.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:46 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:46.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:46 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:46.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:46 smithi091 ceph-mon[30290]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:46.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:46 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:46.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:46 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:46.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:46 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:46.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:46 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:46 smithi120 ceph-mon[30899]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:47.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:46 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:46 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:46 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:46 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:47 smithi091 ceph-mon[30290]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:47.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:47 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:47 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:47 smithi091 ceph-mon[30290]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-14T17:17:47.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:47 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:47 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:47 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:17:47.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:47 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:47 smithi082 ceph-mon[25796]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:47.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:47 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:47 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:47 smithi082 ceph-mon[25796]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-14T17:17:47.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:47 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:47 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:47 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:17:47.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:47 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:47.887 INFO:teuthology.orchestra.run.smithi082.stdout:34359738390 2024-02-14T17:17:47.887 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-14T17:17:48.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:47 smithi120 ceph-mon[30899]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:48.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:47 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:48.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:47 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:48.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:47 smithi120 ceph-mon[30899]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-14T17:17:48.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:47 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:48.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:47 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:48.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:47 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:17:48.056 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:47 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:48.519 INFO:teuthology.orchestra.run.smithi082.stdout:77309411340 2024-02-14T17:17:48.519 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-14T17:17:48.587 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:48.865 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:48 smithi082 ceph-mon[25796]: Adjusting osd_memory_target on smithi091 to 16064M 2024-02-14T17:17:49.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:48 smithi120 ceph-mon[30899]: Adjusting osd_memory_target on smithi091 to 16064M 2024-02-14T17:17:49.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:48 smithi091 ceph-mon[30290]: Adjusting osd_memory_target on smithi091 to 16064M 2024-02-14T17:17:49.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:49 smithi082 ceph-mon[25796]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:49.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:49 smithi082 ceph-mon[25796]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-14T17:17:50.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:49 smithi120 ceph-mon[30899]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:50.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:49 smithi120 ceph-mon[30899]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-14T17:17:50.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:49 smithi091 ceph-mon[30290]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:50.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:49 smithi091 ceph-mon[30290]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-14T17:17:50.330 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:50.331 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:51.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:51 smithi082 ceph-mon[25796]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:51.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:51 smithi120 ceph-mon[30899]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:52.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:51 smithi091 ceph-mon[30290]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:17:53.164 INFO:teuthology.orchestra.run.smithi082.stdout:34359738391 2024-02-14T17:17:53.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:53 smithi091 ceph-mon[30290]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-14T17:17:53.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:53 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:53.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:53 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:53.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:53 smithi082 ceph-mon[25796]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-14T17:17:53.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:53 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:53.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:53 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:53.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:53 smithi120 ceph-mon[30899]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-14T17:17:53.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:53 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:53.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:53 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:53.746 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738390 got 34359738391 for osd.0 2024-02-14T17:17:53.746 DEBUG:teuthology.parallel:result is None 2024-02-14T17:17:54.147 INFO:teuthology.orchestra.run.smithi082.stdout:55834574864 2024-02-14T17:17:54.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:54 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/3641994410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T17:17:54.432 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:54 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/3641994410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T17:17:54.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:54 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/3641994410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T17:17:54.601 INFO:teuthology.orchestra.run.smithi082.stdout:77309411341 2024-02-14T17:17:55.231 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:17:54 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ::ffff:172.21.15.91 - - [14/Feb/2024:17:17:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/1071405469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: Adjusting osd_memory_target on smithi082 to 16064M 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:55.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: Updating smithi082:/etc/ceph/ceph.conf 2024-02-14T17:17:55.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: Updating smithi091:/etc/ceph/ceph.conf 2024-02-14T17:17:55.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: Updating smithi120:/etc/ceph/ceph.conf 2024-02-14T17:17:55.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:55 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/2029864459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T17:17:55.325 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574862 got 55834574864 for osd.1 2024-02-14T17:17:55.325 DEBUG:teuthology.parallel:result is None 2024-02-14T17:17:55.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-14T17:17:55.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/1071405469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T17:17:55.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: Adjusting osd_memory_target on smithi082 to 16064M 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: Updating smithi082:/etc/ceph/ceph.conf 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: Updating smithi091:/etc/ceph/ceph.conf 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: Updating smithi120:/etc/ceph/ceph.conf 2024-02-14T17:17:55.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:55 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/2029864459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T17:17:55.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-14T17:17:55.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/1071405469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T17:17:55.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:55.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:55.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:17:55.555 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: Adjusting osd_memory_target on smithi082 to 16064M 2024-02-14T17:17:55.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:55.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:17:55.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:17:55.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: Updating smithi082:/etc/ceph/ceph.conf 2024-02-14T17:17:55.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: Updating smithi091:/etc/ceph/ceph.conf 2024-02-14T17:17:55.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: Updating smithi120:/etc/ceph/ceph.conf 2024-02-14T17:17:55.556 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:55 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/2029864459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T17:17:55.849 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411340 got 77309411341 for osd.2 2024-02-14T17:17:55.849 DEBUG:teuthology.parallel:result is None 2024-02-14T17:17:55.849 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-14T17:17:55.850 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-14T17:17:56.092 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:56.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:17:56.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:17:56.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:17:56.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:17:56.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Updating smithi091:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:17:56.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:17:56.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:56 smithi082 ceph-mon[25796]: Deploying daemon alertmanager.a on smithi082 2024-02-14T17:17:56.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:17:56.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:17:56.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Updating smithi091:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.807 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:56 smithi120 ceph-mon[30899]: Deploying daemon alertmanager.a on smithi082 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Updating smithi091:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Updating smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Updating smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Updating smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:17:56.855 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:56 smithi091 ceph-mon[30290]: Deploying daemon alertmanager.a on smithi082 2024-02-14T17:17:57.906 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:57 smithi082 ceph-mon[25796]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:17:58.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:57 smithi120 ceph-mon[30899]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:17:58.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:57 smithi091 ceph-mon[30290]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:17:58.220 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:17:58.223 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-02-14T17:17:59.350 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-02-14T17:17:57.855086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96612,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126556,"statfs":{"total":287972524032,"available":287873593344,"internally_reserved":0,"allocated":3158016,"data_stored":2100345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003006"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842634+0000","last_change":"2024-02-14T17:17:21.921850+0000","last_active":"2024-02-14T17:17:43.842634+0000","last_peered":"2024-02-14T17:17:43.842634+0000","last_clean":"2024-02-14T17:17:43.842634+0000","last_became_active":"2024-02-14T17:17:21.921386+0000","last_became_peered":"2024-02-14T17:17:21.921386+0000","last_unstale":"2024-02-14T17:17:43.842634+0000","last_undegraded":"2024-02-14T17:17:43.842634+0000","last_fullsized":"2024-02-14T17:17:43.842634+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:15:41.278032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842481+0000","last_change":"2024-02-14T17:17:19.905875+0000","last_active":"2024-02-14T17:17:43.842481+0000","last_peered":"2024-02-14T17:17:43.842481+0000","last_clean":"2024-02-14T17:17:43.842481+0000","last_became_active":"2024-02-14T17:17:19.905480+0000","last_became_peered":"2024-02-14T17:17:19.905480+0000","last_unstale":"2024-02-14T17:17:43.842481+0000","last_undegraded":"2024-02-14T17:17:43.842481+0000","last_fullsized":"2024-02-14T17:17:43.842481+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:28:14.250316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846463+0000","last_change":"2024-02-14T17:17:15.884394+0000","last_active":"2024-02-14T17:17:43.846463+0000","last_peered":"2024-02-14T17:17:43.846463+0000","last_clean":"2024-02-14T17:17:43.846463+0000","last_became_active":"2024-02-14T17:17:15.883878+0000","last_became_peered":"2024-02-14T17:17:15.883878+0000","last_unstale":"2024-02-14T17:17:43.846463+0000","last_undegraded":"2024-02-14T17:17:43.846463+0000","last_fullsized":"2024-02-14T17:17:43.846463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:49:21.806559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844657+0000","last_change":"2024-02-14T17:17:17.894753+0000","last_active":"2024-02-14T17:17:43.844657+0000","last_peered":"2024-02-14T17:17:43.844657+0000","last_clean":"2024-02-14T17:17:43.844657+0000","last_became_active":"2024-02-14T17:17:17.894487+0000","last_became_peered":"2024-02-14T17:17:17.894487+0000","last_unstale":"2024-02-14T17:17:43.844657+0000","last_undegraded":"2024-02-14T17:17:43.844657+0000","last_fullsized":"2024-02-14T17:17:43.844657+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:20:48.068097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841649+0000","last_change":"2024-02-14T17:17:21.919780+0000","last_active":"2024-02-14T17:17:43.841649+0000","last_peered":"2024-02-14T17:17:43.841649+0000","last_clean":"2024-02-14T17:17:43.841649+0000","last_became_active":"2024-02-14T17:17:21.919402+0000","last_became_peered":"2024-02-14T17:17:21.919402+0000","last_unstale":"2024-02-14T17:17:43.841649+0000","last_undegraded":"2024-02-14T17:17:43.841649+0000","last_fullsized":"2024-02-14T17:17:43.841649+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:58:10.934431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842327+0000","last_change":"2024-02-14T17:17:19.910232+0000","last_active":"2024-02-14T17:17:43.842327+0000","last_peered":"2024-02-14T17:17:43.842327+0000","last_clean":"2024-02-14T17:17:43.842327+0000","last_became_active":"2024-02-14T17:17:19.909926+0000","last_became_peered":"2024-02-14T17:17:19.909926+0000","last_unstale":"2024-02-14T17:17:43.842327+0000","last_undegraded":"2024-02-14T17:17:43.842327+0000","last_fullsized":"2024-02-14T17:17:43.842327+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:46:43.286543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844931+0000","last_change":"2024-02-14T17:17:15.883865+0000","last_active":"2024-02-14T17:17:43.844931+0000","last_peered":"2024-02-14T17:17:43.844931+0000","last_clean":"2024-02-14T17:17:43.844931+0000","last_became_active":"2024-02-14T17:17:15.883419+0000","last_became_peered":"2024-02-14T17:17:15.883419+0000","last_unstale":"2024-02-14T17:17:43.844931+0000","last_undegraded":"2024-02-14T17:17:43.844931+0000","last_fullsized":"2024-02-14T17:17:43.844931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:36:02.857579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846034+0000","last_change":"2024-02-14T17:17:17.895152+0000","last_active":"2024-02-14T17:17:43.846034+0000","last_peered":"2024-02-14T17:17:43.846034+0000","last_clean":"2024-02-14T17:17:43.846034+0000","last_became_active":"2024-02-14T17:17:17.894425+0000","last_became_peered":"2024-02-14T17:17:17.894425+0000","last_unstale":"2024-02-14T17:17:43.846034+0000","last_undegraded":"2024-02-14T17:17:43.846034+0000","last_fullsized":"2024-02-14T17:17:43.846034+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:48:38.940957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843540+0000","last_change":"2024-02-14T17:17:21.920558+0000","last_active":"2024-02-14T17:17:43.843540+0000","last_peered":"2024-02-14T17:17:43.843540+0000","last_clean":"2024-02-14T17:17:43.843540+0000","last_became_active":"2024-02-14T17:17:21.920280+0000","last_became_peered":"2024-02-14T17:17:21.920280+0000","last_unstale":"2024-02-14T17:17:43.843540+0000","last_undegraded":"2024-02-14T17:17:43.843540+0000","last_fullsized":"2024-02-14T17:17:43.843540+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:50:52.929464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842813+0000","last_change":"2024-02-14T17:17:19.909807+0000","last_active":"2024-02-14T17:17:43.842813+0000","last_peered":"2024-02-14T17:17:43.842813+0000","last_clean":"2024-02-14T17:17:43.842813+0000","last_became_active":"2024-02-14T17:17:19.908926+0000","last_became_peered":"2024-02-14T17:17:19.908926+0000","last_unstale":"2024-02-14T17:17:43.842813+0000","last_undegraded":"2024-02-14T17:17:43.842813+0000","last_fullsized":"2024-02-14T17:17:43.842813+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:02:31.200331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845398+0000","last_change":"2024-02-14T17:17:15.888439+0000","last_active":"2024-02-14T17:17:43.845398+0000","last_peered":"2024-02-14T17:17:43.845398+0000","last_clean":"2024-02-14T17:17:43.845398+0000","last_became_active":"2024-02-14T17:17:15.888083+0000","last_became_peered":"2024-02-14T17:17:15.888083+0000","last_unstale":"2024-02-14T17:17:43.845398+0000","last_undegraded":"2024-02-14T17:17:43.845398+0000","last_fullsized":"2024-02-14T17:17:43.845398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:10:33.711513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844400+0000","last_change":"2024-02-14T17:17:17.892428+0000","last_active":"2024-02-14T17:17:43.844400+0000","last_peered":"2024-02-14T17:17:43.844400+0000","last_clean":"2024-02-14T17:17:43.844400+0000","last_became_active":"2024-02-14T17:17:17.891911+0000","last_became_peered":"2024-02-14T17:17:17.891911+0000","last_unstale":"2024-02-14T17:17:43.844400+0000","last_undegraded":"2024-02-14T17:17:43.844400+0000","last_fullsized":"2024-02-14T17:17:43.844400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:26:03.087984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842722+0000","last_change":"2024-02-14T17:17:21.922077+0000","last_active":"2024-02-14T17:17:43.842722+0000","last_peered":"2024-02-14T17:17:43.842722+0000","last_clean":"2024-02-14T17:17:43.842722+0000","last_became_active":"2024-02-14T17:17:21.921509+0000","last_became_peered":"2024-02-14T17:17:21.921509+0000","last_unstale":"2024-02-14T17:17:43.842722+0000","last_undegraded":"2024-02-14T17:17:43.842722+0000","last_fullsized":"2024-02-14T17:17:43.842722+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:01:04.569431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842829+0000","last_change":"2024-02-14T17:17:19.909867+0000","last_active":"2024-02-14T17:17:43.842829+0000","last_peered":"2024-02-14T17:17:43.842829+0000","last_clean":"2024-02-14T17:17:43.842829+0000","last_became_active":"2024-02-14T17:17:19.909274+0000","last_became_peered":"2024-02-14T17:17:19.909274+0000","last_unstale":"2024-02-14T17:17:43.842829+0000","last_undegraded":"2024-02-14T17:17:43.842829+0000","last_fullsized":"2024-02-14T17:17:43.842829+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:29:50.359669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845315+0000","last_change":"2024-02-14T17:17:15.887941+0000","last_active":"2024-02-14T17:17:43.845315+0000","last_peered":"2024-02-14T17:17:43.845315+0000","last_clean":"2024-02-14T17:17:43.845315+0000","last_became_active":"2024-02-14T17:17:15.886905+0000","last_became_peered":"2024-02-14T17:17:15.886905+0000","last_unstale":"2024-02-14T17:17:43.845315+0000","last_undegraded":"2024-02-14T17:17:43.845315+0000","last_fullsized":"2024-02-14T17:17:43.845315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:39:48.099305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843512+0000","last_change":"2024-02-14T17:17:17.898290+0000","last_active":"2024-02-14T17:17:43.843512+0000","last_peered":"2024-02-14T17:17:43.843512+0000","last_clean":"2024-02-14T17:17:43.843512+0000","last_became_active":"2024-02-14T17:17:17.897984+0000","last_became_peered":"2024-02-14T17:17:17.897984+0000","last_unstale":"2024-02-14T17:17:43.843512+0000","last_undegraded":"2024-02-14T17:17:43.843512+0000","last_fullsized":"2024-02-14T17:17:43.843512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:21:05.388054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'2","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844035+0000","last_change":"2024-02-14T17:17:21.921435+0000","last_active":"2024-02-14T17:17:43.844035+0000","last_peered":"2024-02-14T17:17:43.844035+0000","last_clean":"2024-02-14T17:17:43.844035+0000","last_became_active":"2024-02-14T17:17:21.920977+0000","last_became_peered":"2024-02-14T17:17:21.920977+0000","last_unstale":"2024-02-14T17:17:43.844035+0000","last_undegraded":"2024-02-14T17:17:43.844035+0000","last_fullsized":"2024-02-14T17:17:43.844035+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:47:13.144506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843094+0000","last_change":"2024-02-14T17:17:19.906153+0000","last_active":"2024-02-14T17:17:43.843094+0000","last_peered":"2024-02-14T17:17:43.843094+0000","last_clean":"2024-02-14T17:17:43.843094+0000","last_became_active":"2024-02-14T17:17:19.905424+0000","last_became_peered":"2024-02-14T17:17:19.905424+0000","last_unstale":"2024-02-14T17:17:43.843094+0000","last_undegraded":"2024-02-14T17:17:43.843094+0000","last_fullsized":"2024-02-14T17:17:43.843094+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:27:42.210427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845451+0000","last_change":"2024-02-14T17:17:15.886593+0000","last_active":"2024-02-14T17:17:43.845451+0000","last_peered":"2024-02-14T17:17:43.845451+0000","last_clean":"2024-02-14T17:17:43.845451+0000","last_became_active":"2024-02-14T17:17:15.886184+0000","last_became_peered":"2024-02-14T17:17:15.886184+0000","last_unstale":"2024-02-14T17:17:43.845451+0000","last_undegraded":"2024-02-14T17:17:43.845451+0000","last_fullsized":"2024-02-14T17:17:43.845451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:29:36.526730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845150+0000","last_change":"2024-02-14T17:17:17.897527+0000","last_active":"2024-02-14T17:17:43.845150+0000","last_peered":"2024-02-14T17:17:43.845150+0000","last_clean":"2024-02-14T17:17:43.845150+0000","last_became_active":"2024-02-14T17:17:17.896690+0000","last_became_peered":"2024-02-14T17:17:17.896690+0000","last_unstale":"2024-02-14T17:17:43.845150+0000","last_undegraded":"2024-02-14T17:17:43.845150+0000","last_fullsized":"2024-02-14T17:17:43.845150+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:22:22.689895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844274+0000","last_change":"2024-02-14T17:17:21.926339+0000","last_active":"2024-02-14T17:17:43.844274+0000","last_peered":"2024-02-14T17:17:43.844274+0000","last_clean":"2024-02-14T17:17:43.844274+0000","last_became_active":"2024-02-14T17:17:21.925921+0000","last_became_peered":"2024-02-14T17:17:21.925921+0000","last_unstale":"2024-02-14T17:17:43.844274+0000","last_undegraded":"2024-02-14T17:17:43.844274+0000","last_fullsized":"2024-02-14T17:17:43.844274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:16:56.635263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844406+0000","last_change":"2024-02-14T17:17:19.906688+0000","last_active":"2024-02-14T17:17:43.844406+0000","last_peered":"2024-02-14T17:17:43.844406+0000","last_clean":"2024-02-14T17:17:43.844406+0000","last_became_active":"2024-02-14T17:17:19.906016+0000","last_became_peered":"2024-02-14T17:17:19.906016+0000","last_unstale":"2024-02-14T17:17:43.844406+0000","last_undegraded":"2024-02-14T17:17:43.844406+0000","last_fullsized":"2024-02-14T17:17:43.844406+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:44:51.700864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843225+0000","last_change":"2024-02-14T17:17:15.884881+0000","last_active":"2024-02-14T17:17:43.843225+0000","last_peered":"2024-02-14T17:17:43.843225+0000","last_clean":"2024-02-14T17:17:43.843225+0000","last_became_active":"2024-02-14T17:17:15.884474+0000","last_became_peered":"2024-02-14T17:17:15.884474+0000","last_unstale":"2024-02-14T17:17:43.843225+0000","last_undegraded":"2024-02-14T17:17:43.843225+0000","last_fullsized":"2024-02-14T17:17:43.843225+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:24:40.731728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845127+0000","last_change":"2024-02-14T17:17:17.896663+0000","last_active":"2024-02-14T17:17:43.845127+0000","last_peered":"2024-02-14T17:17:43.845127+0000","last_clean":"2024-02-14T17:17:43.845127+0000","last_became_active":"2024-02-14T17:17:17.896034+0000","last_became_peered":"2024-02-14T17:17:17.896034+0000","last_unstale":"2024-02-14T17:17:43.845127+0000","last_undegraded":"2024-02-14T17:17:43.845127+0000","last_fullsized":"2024-02-14T17:17:43.845127+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:43:40.345511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844020+0000","last_change":"2024-02-14T17:17:21.926167+0000","last_active":"2024-02-14T17:17:43.844020+0000","last_peered":"2024-02-14T17:17:43.844020+0000","last_clean":"2024-02-14T17:17:43.844020+0000","last_became_active":"2024-02-14T17:17:21.925624+0000","last_became_peered":"2024-02-14T17:17:21.925624+0000","last_unstale":"2024-02-14T17:17:43.844020+0000","last_undegraded":"2024-02-14T17:17:43.844020+0000","last_fullsized":"2024-02-14T17:17:43.844020+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:55:25.645101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929716+0000","last_change":"2024-02-14T17:17:19.909069+0000","last_active":"2024-02-14T17:17:45.929716+0000","last_peered":"2024-02-14T17:17:45.929716+0000","last_clean":"2024-02-14T17:17:45.929716+0000","last_became_active":"2024-02-14T17:17:19.908326+0000","last_became_peered":"2024-02-14T17:17:19.908326+0000","last_unstale":"2024-02-14T17:17:45.929716+0000","last_undegraded":"2024-02-14T17:17:45.929716+0000","last_fullsized":"2024-02-14T17:17:45.929716+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:57:01.243136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841845+0000","last_change":"2024-02-14T17:17:15.885539+0000","last_active":"2024-02-14T17:17:43.841845+0000","last_peered":"2024-02-14T17:17:43.841845+0000","last_clean":"2024-02-14T17:17:43.841845+0000","last_became_active":"2024-02-14T17:17:15.885267+0000","last_became_peered":"2024-02-14T17:17:15.885267+0000","last_unstale":"2024-02-14T17:17:43.841845+0000","last_undegraded":"2024-02-14T17:17:43.841845+0000","last_fullsized":"2024-02-14T17:17:43.841845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:45:52.655197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841450+0000","last_change":"2024-02-14T17:17:17.892510+0000","last_active":"2024-02-14T17:17:43.841450+0000","last_peered":"2024-02-14T17:17:43.841450+0000","last_clean":"2024-02-14T17:17:43.841450+0000","last_became_active":"2024-02-14T17:17:17.892143+0000","last_became_peered":"2024-02-14T17:17:17.892143+0000","last_unstale":"2024-02-14T17:17:43.841450+0000","last_undegraded":"2024-02-14T17:17:43.841450+0000","last_fullsized":"2024-02-14T17:17:43.841450+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:50:58.816774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844450+0000","last_change":"2024-02-14T17:17:21.923968+0000","last_active":"2024-02-14T17:17:43.844450+0000","last_peered":"2024-02-14T17:17:43.844450+0000","last_clean":"2024-02-14T17:17:43.844450+0000","last_became_active":"2024-02-14T17:17:21.921806+0000","last_became_peered":"2024-02-14T17:17:21.921806+0000","last_unstale":"2024-02-14T17:17:43.844450+0000","last_undegraded":"2024-02-14T17:17:43.844450+0000","last_fullsized":"2024-02-14T17:17:43.844450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:30:28.966847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844298+0000","last_change":"2024-02-14T17:17:19.910353+0000","last_active":"2024-02-14T17:17:43.844298+0000","last_peered":"2024-02-14T17:17:43.844298+0000","last_clean":"2024-02-14T17:17:43.844298+0000","last_became_active":"2024-02-14T17:17:19.910020+0000","last_became_peered":"2024-02-14T17:17:19.910020+0000","last_unstale":"2024-02-14T17:17:43.844298+0000","last_undegraded":"2024-02-14T17:17:43.844298+0000","last_fullsized":"2024-02-14T17:17:43.844298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:47:15.960147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842445+0000","last_change":"2024-02-14T17:17:15.886401+0000","last_active":"2024-02-14T17:17:43.842445+0000","last_peered":"2024-02-14T17:17:43.842445+0000","last_clean":"2024-02-14T17:17:43.842445+0000","last_became_active":"2024-02-14T17:17:15.885918+0000","last_became_peered":"2024-02-14T17:17:15.885918+0000","last_unstale":"2024-02-14T17:17:43.842445+0000","last_undegraded":"2024-02-14T17:17:43.842445+0000","last_fullsized":"2024-02-14T17:17:43.842445+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:55:40.012434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841501+0000","last_change":"2024-02-14T17:17:17.889690+0000","last_active":"2024-02-14T17:17:43.841501+0000","last_peered":"2024-02-14T17:17:43.841501+0000","last_clean":"2024-02-14T17:17:43.841501+0000","last_became_active":"2024-02-14T17:17:17.889371+0000","last_became_peered":"2024-02-14T17:17:17.889371+0000","last_unstale":"2024-02-14T17:17:43.841501+0000","last_undegraded":"2024-02-14T17:17:43.841501+0000","last_fullsized":"2024-02-14T17:17:43.841501+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:46:42.045637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842501+0000","last_change":"2024-02-14T17:17:21.924013+0000","last_active":"2024-02-14T17:17:43.842501+0000","last_peered":"2024-02-14T17:17:43.842501+0000","last_clean":"2024-02-14T17:17:43.842501+0000","last_became_active":"2024-02-14T17:17:21.922231+0000","last_became_peered":"2024-02-14T17:17:21.922231+0000","last_unstale":"2024-02-14T17:17:43.842501+0000","last_undegraded":"2024-02-14T17:17:43.842501+0000","last_fullsized":"2024-02-14T17:17:43.842501+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:34:11.654557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841427+0000","last_change":"2024-02-14T17:17:19.911419+0000","last_active":"2024-02-14T17:17:43.841427+0000","last_peered":"2024-02-14T17:17:43.841427+0000","last_clean":"2024-02-14T17:17:43.841427+0000","last_became_active":"2024-02-14T17:17:19.910804+0000","last_became_peered":"2024-02-14T17:17:19.910804+0000","last_unstale":"2024-02-14T17:17:43.841427+0000","last_undegraded":"2024-02-14T17:17:43.841427+0000","last_fullsized":"2024-02-14T17:17:43.841427+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:32:34.823529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845372+0000","last_change":"2024-02-14T17:17:15.887700+0000","last_active":"2024-02-14T17:17:43.845372+0000","last_peered":"2024-02-14T17:17:43.845372+0000","last_clean":"2024-02-14T17:17:43.845372+0000","last_became_active":"2024-02-14T17:17:15.886138+0000","last_became_peered":"2024-02-14T17:17:15.886138+0000","last_unstale":"2024-02-14T17:17:43.845372+0000","last_undegraded":"2024-02-14T17:17:43.845372+0000","last_fullsized":"2024-02-14T17:17:43.845372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:55:35.680409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845406+0000","last_change":"2024-02-14T17:17:17.893826+0000","last_active":"2024-02-14T17:17:43.845406+0000","last_peered":"2024-02-14T17:17:43.845406+0000","last_clean":"2024-02-14T17:17:43.845406+0000","last_became_active":"2024-02-14T17:17:17.893412+0000","last_became_peered":"2024-02-14T17:17:17.893412+0000","last_unstale":"2024-02-14T17:17:43.845406+0000","last_undegraded":"2024-02-14T17:17:43.845406+0000","last_fullsized":"2024-02-14T17:17:43.845406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:44:26.910135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842392+0000","last_change":"2024-02-14T17:17:21.921625+0000","last_active":"2024-02-14T17:17:43.842392+0000","last_peered":"2024-02-14T17:17:43.842392+0000","last_clean":"2024-02-14T17:17:43.842392+0000","last_became_active":"2024-02-14T17:17:21.920249+0000","last_became_peered":"2024-02-14T17:17:21.920249+0000","last_unstale":"2024-02-14T17:17:43.842392+0000","last_undegraded":"2024-02-14T17:17:43.842392+0000","last_fullsized":"2024-02-14T17:17:43.842392+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:28:35.624949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841607+0000","last_change":"2024-02-14T17:17:19.907668+0000","last_active":"2024-02-14T17:17:43.841607+0000","last_peered":"2024-02-14T17:17:43.841607+0000","last_clean":"2024-02-14T17:17:43.841607+0000","last_became_active":"2024-02-14T17:17:19.907461+0000","last_became_peered":"2024-02-14T17:17:19.907461+0000","last_unstale":"2024-02-14T17:17:43.841607+0000","last_undegraded":"2024-02-14T17:17:43.841607+0000","last_fullsized":"2024-02-14T17:17:43.841607+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:19:28.580967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845152+0000","last_change":"2024-02-14T17:17:15.886720+0000","last_active":"2024-02-14T17:17:43.845152+0000","last_peered":"2024-02-14T17:17:43.845152+0000","last_clean":"2024-02-14T17:17:43.845152+0000","last_became_active":"2024-02-14T17:17:15.886129+0000","last_became_peered":"2024-02-14T17:17:15.886129+0000","last_unstale":"2024-02-14T17:17:43.845152+0000","last_undegraded":"2024-02-14T17:17:43.845152+0000","last_fullsized":"2024-02-14T17:17:43.845152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:05:56.680074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846235+0000","last_change":"2024-02-14T17:17:17.897144+0000","last_active":"2024-02-14T17:17:43.846235+0000","last_peered":"2024-02-14T17:17:43.846235+0000","last_clean":"2024-02-14T17:17:43.846235+0000","last_became_active":"2024-02-14T17:17:17.896665+0000","last_became_peered":"2024-02-14T17:17:17.896665+0000","last_unstale":"2024-02-14T17:17:43.846235+0000","last_undegraded":"2024-02-14T17:17:43.846235+0000","last_fullsized":"2024-02-14T17:17:43.846235+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:49:25.852754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843686+0000","last_change":"2024-02-14T17:17:21.920499+0000","last_active":"2024-02-14T17:17:43.843686+0000","last_peered":"2024-02-14T17:17:43.843686+0000","last_clean":"2024-02-14T17:17:43.843686+0000","last_became_active":"2024-02-14T17:17:21.920204+0000","last_became_peered":"2024-02-14T17:17:21.920204+0000","last_unstale":"2024-02-14T17:17:43.843686+0000","last_undegraded":"2024-02-14T17:17:43.843686+0000","last_fullsized":"2024-02-14T17:17:43.843686+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:02:29.697540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842893+0000","last_change":"2024-02-14T17:17:19.909249+0000","last_active":"2024-02-14T17:17:43.842893+0000","last_peered":"2024-02-14T17:17:43.842893+0000","last_clean":"2024-02-14T17:17:43.842893+0000","last_became_active":"2024-02-14T17:17:19.908670+0000","last_became_peered":"2024-02-14T17:17:19.908670+0000","last_unstale":"2024-02-14T17:17:43.842893+0000","last_undegraded":"2024-02-14T17:17:43.842893+0000","last_fullsized":"2024-02-14T17:17:43.842893+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:09:16.040128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845394+0000","last_change":"2024-02-14T17:17:15.888261+0000","last_active":"2024-02-14T17:17:43.845394+0000","last_peered":"2024-02-14T17:17:43.845394+0000","last_clean":"2024-02-14T17:17:43.845394+0000","last_became_active":"2024-02-14T17:17:15.887978+0000","last_became_peered":"2024-02-14T17:17:15.887978+0000","last_unstale":"2024-02-14T17:17:43.845394+0000","last_undegraded":"2024-02-14T17:17:43.845394+0000","last_fullsized":"2024-02-14T17:17:43.845394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:13.031412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844889+0000","last_change":"2024-02-14T17:17:17.892929+0000","last_active":"2024-02-14T17:17:43.844889+0000","last_peered":"2024-02-14T17:17:43.844889+0000","last_clean":"2024-02-14T17:17:43.844889+0000","last_became_active":"2024-02-14T17:17:17.891909+0000","last_became_peered":"2024-02-14T17:17:17.891909+0000","last_unstale":"2024-02-14T17:17:43.844889+0000","last_undegraded":"2024-02-14T17:17:43.844889+0000","last_fullsized":"2024-02-14T17:17:43.844889+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:38:00.734469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842863+0000","last_change":"2024-02-14T17:17:21.920505+0000","last_active":"2024-02-14T17:17:43.842863+0000","last_peered":"2024-02-14T17:17:43.842863+0000","last_clean":"2024-02-14T17:17:43.842863+0000","last_became_active":"2024-02-14T17:17:21.920171+0000","last_became_peered":"2024-02-14T17:17:21.920171+0000","last_unstale":"2024-02-14T17:17:43.842863+0000","last_undegraded":"2024-02-14T17:17:43.842863+0000","last_fullsized":"2024-02-14T17:17:43.842863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:44:26.244773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842180+0000","last_change":"2024-02-14T17:17:19.911072+0000","last_active":"2024-02-14T17:17:43.842180+0000","last_peered":"2024-02-14T17:17:43.842180+0000","last_clean":"2024-02-14T17:17:43.842180+0000","last_became_active":"2024-02-14T17:17:19.910665+0000","last_became_peered":"2024-02-14T17:17:19.910665+0000","last_unstale":"2024-02-14T17:17:43.842180+0000","last_undegraded":"2024-02-14T17:17:43.842180+0000","last_fullsized":"2024-02-14T17:17:43.842180+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:04:28.401412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844172+0000","last_change":"2024-02-14T17:17:15.886827+0000","last_active":"2024-02-14T17:17:43.844172+0000","last_peered":"2024-02-14T17:17:43.844172+0000","last_clean":"2024-02-14T17:17:43.844172+0000","last_became_active":"2024-02-14T17:17:15.886068+0000","last_became_peered":"2024-02-14T17:17:15.886068+0000","last_unstale":"2024-02-14T17:17:43.844172+0000","last_undegraded":"2024-02-14T17:17:43.844172+0000","last_fullsized":"2024-02-14T17:17:43.844172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:27:32.304159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843655+0000","last_change":"2024-02-14T17:17:17.897817+0000","last_active":"2024-02-14T17:17:43.843655+0000","last_peered":"2024-02-14T17:17:43.843655+0000","last_clean":"2024-02-14T17:17:43.843655+0000","last_became_active":"2024-02-14T17:17:17.897317+0000","last_became_peered":"2024-02-14T17:17:17.897317+0000","last_unstale":"2024-02-14T17:17:43.843655+0000","last_undegraded":"2024-02-14T17:17:43.843655+0000","last_fullsized":"2024-02-14T17:17:43.843655+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:45:52.526374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844914+0000","last_change":"2024-02-14T17:17:21.925924+0000","last_active":"2024-02-14T17:17:43.844914+0000","last_peered":"2024-02-14T17:17:43.844914+0000","last_clean":"2024-02-14T17:17:43.844914+0000","last_became_active":"2024-02-14T17:17:21.925513+0000","last_became_peered":"2024-02-14T17:17:21.925513+0000","last_unstale":"2024-02-14T17:17:43.844914+0000","last_undegraded":"2024-02-14T17:17:43.844914+0000","last_fullsized":"2024-02-14T17:17:43.844914+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:36:29.218922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929710+0000","last_change":"2024-02-14T17:17:19.908714+0000","last_active":"2024-02-14T17:17:45.929710+0000","last_peered":"2024-02-14T17:17:45.929710+0000","last_clean":"2024-02-14T17:17:45.929710+0000","last_became_active":"2024-02-14T17:17:19.908482+0000","last_became_peered":"2024-02-14T17:17:19.908482+0000","last_unstale":"2024-02-14T17:17:45.929710+0000","last_undegraded":"2024-02-14T17:17:45.929710+0000","last_fullsized":"2024-02-14T17:17:45.929710+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:35:22.151972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845530+0000","last_change":"2024-02-14T17:17:15.887365+0000","last_active":"2024-02-14T17:17:43.845530+0000","last_peered":"2024-02-14T17:17:43.845530+0000","last_clean":"2024-02-14T17:17:43.845530+0000","last_became_active":"2024-02-14T17:17:15.886493+0000","last_became_peered":"2024-02-14T17:17:15.886493+0000","last_unstale":"2024-02-14T17:17:43.845530+0000","last_undegraded":"2024-02-14T17:17:43.845530+0000","last_fullsized":"2024-02-14T17:17:43.845530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:42:40.980118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842433+0000","last_change":"2024-02-14T17:17:17.889757+0000","last_active":"2024-02-14T17:17:43.842433+0000","last_peered":"2024-02-14T17:17:43.842433+0000","last_clean":"2024-02-14T17:17:43.842433+0000","last_became_active":"2024-02-14T17:17:17.889413+0000","last_became_peered":"2024-02-14T17:17:17.889413+0000","last_unstale":"2024-02-14T17:17:43.842433+0000","last_undegraded":"2024-02-14T17:17:43.842433+0000","last_fullsized":"2024-02-14T17:17:43.842433+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:18:26.829865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844012+0000","last_change":"2024-02-14T17:17:21.922052+0000","last_active":"2024-02-14T17:17:43.844012+0000","last_peered":"2024-02-14T17:17:43.844012+0000","last_clean":"2024-02-14T17:17:43.844012+0000","last_became_active":"2024-02-14T17:17:21.920969+0000","last_became_peered":"2024-02-14T17:17:21.920969+0000","last_unstale":"2024-02-14T17:17:43.844012+0000","last_undegraded":"2024-02-14T17:17:43.844012+0000","last_fullsized":"2024-02-14T17:17:43.844012+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:04:17.155323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929684+0000","last_change":"2024-02-14T17:17:19.908580+0000","last_active":"2024-02-14T17:17:45.929684+0000","last_peered":"2024-02-14T17:17:45.929684+0000","last_clean":"2024-02-14T17:17:45.929684+0000","last_became_active":"2024-02-14T17:17:19.908385+0000","last_became_peered":"2024-02-14T17:17:19.908385+0000","last_unstale":"2024-02-14T17:17:45.929684+0000","last_undegraded":"2024-02-14T17:17:45.929684+0000","last_fullsized":"2024-02-14T17:17:45.929684+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:39:04.978103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843940+0000","last_change":"2024-02-14T17:17:15.886323+0000","last_active":"2024-02-14T17:17:43.843940+0000","last_peered":"2024-02-14T17:17:43.843940+0000","last_clean":"2024-02-14T17:17:43.843940+0000","last_became_active":"2024-02-14T17:17:15.885955+0000","last_became_peered":"2024-02-14T17:17:15.885955+0000","last_unstale":"2024-02-14T17:17:43.843940+0000","last_undegraded":"2024-02-14T17:17:43.843940+0000","last_fullsized":"2024-02-14T17:17:43.843940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:13:22.299558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843121+0000","last_change":"2024-02-14T17:17:17.894466+0000","last_active":"2024-02-14T17:17:43.843121+0000","last_peered":"2024-02-14T17:17:43.843121+0000","last_clean":"2024-02-14T17:17:43.843121+0000","last_became_active":"2024-02-14T17:17:17.893598+0000","last_became_peered":"2024-02-14T17:17:17.893598+0000","last_unstale":"2024-02-14T17:17:43.843121+0000","last_undegraded":"2024-02-14T17:17:43.843121+0000","last_fullsized":"2024-02-14T17:17:43.843121+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:17:56.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844357+0000","last_change":"2024-02-14T17:17:21.921966+0000","last_active":"2024-02-14T17:17:43.844357+0000","last_peered":"2024-02-14T17:17:43.844357+0000","last_clean":"2024-02-14T17:17:43.844357+0000","last_became_active":"2024-02-14T17:17:21.921232+0000","last_became_peered":"2024-02-14T17:17:21.921232+0000","last_unstale":"2024-02-14T17:17:43.844357+0000","last_undegraded":"2024-02-14T17:17:43.844357+0000","last_fullsized":"2024-02-14T17:17:43.844357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:21:18.438708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845754+0000","last_change":"2024-02-14T17:17:19.910485+0000","last_active":"2024-02-14T17:17:43.845754+0000","last_peered":"2024-02-14T17:17:43.845754+0000","last_clean":"2024-02-14T17:17:43.845754+0000","last_became_active":"2024-02-14T17:17:19.910143+0000","last_became_peered":"2024-02-14T17:17:19.910143+0000","last_unstale":"2024-02-14T17:17:43.845754+0000","last_undegraded":"2024-02-14T17:17:43.845754+0000","last_fullsized":"2024-02-14T17:17:43.845754+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:43:50.628165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842293+0000","last_change":"2024-02-14T17:17:15.886456+0000","last_active":"2024-02-14T17:17:43.842293+0000","last_peered":"2024-02-14T17:17:43.842293+0000","last_clean":"2024-02-14T17:17:43.842293+0000","last_became_active":"2024-02-14T17:17:15.886006+0000","last_became_peered":"2024-02-14T17:17:15.886006+0000","last_unstale":"2024-02-14T17:17:43.842293+0000","last_undegraded":"2024-02-14T17:17:43.842293+0000","last_fullsized":"2024-02-14T17:17:43.842293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:01:37.200369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841888+0000","last_change":"2024-02-14T17:17:17.895636+0000","last_active":"2024-02-14T17:17:43.841888+0000","last_peered":"2024-02-14T17:17:43.841888+0000","last_clean":"2024-02-14T17:17:43.841888+0000","last_became_active":"2024-02-14T17:17:17.895345+0000","last_became_peered":"2024-02-14T17:17:17.895345+0000","last_unstale":"2024-02-14T17:17:43.841888+0000","last_undegraded":"2024-02-14T17:17:43.841888+0000","last_fullsized":"2024-02-14T17:17:43.841888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:14:36.840742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845496+0000","last_change":"2024-02-14T17:17:21.926444+0000","last_active":"2024-02-14T17:17:43.845496+0000","last_peered":"2024-02-14T17:17:43.845496+0000","last_clean":"2024-02-14T17:17:43.845496+0000","last_became_active":"2024-02-14T17:17:21.926054+0000","last_became_peered":"2024-02-14T17:17:21.926054+0000","last_unstale":"2024-02-14T17:17:43.845496+0000","last_undegraded":"2024-02-14T17:17:43.845496+0000","last_fullsized":"2024-02-14T17:17:43.845496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:48:44.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844386+0000","last_change":"2024-02-14T17:17:19.911233+0000","last_active":"2024-02-14T17:17:43.844386+0000","last_peered":"2024-02-14T17:17:43.844386+0000","last_clean":"2024-02-14T17:17:43.844386+0000","last_became_active":"2024-02-14T17:17:19.910829+0000","last_became_peered":"2024-02-14T17:17:19.910829+0000","last_unstale":"2024-02-14T17:17:43.844386+0000","last_undegraded":"2024-02-14T17:17:43.844386+0000","last_fullsized":"2024-02-14T17:17:43.844386+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:48:25.914208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842659+0000","last_change":"2024-02-14T17:17:15.888379+0000","last_active":"2024-02-14T17:17:43.842659+0000","last_peered":"2024-02-14T17:17:43.842659+0000","last_clean":"2024-02-14T17:17:43.842659+0000","last_became_active":"2024-02-14T17:17:15.886610+0000","last_became_peered":"2024-02-14T17:17:15.886610+0000","last_unstale":"2024-02-14T17:17:43.842659+0000","last_undegraded":"2024-02-14T17:17:43.842659+0000","last_fullsized":"2024-02-14T17:17:43.842659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:18:50.359270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842770+0000","last_change":"2024-02-14T17:17:17.897174+0000","last_active":"2024-02-14T17:17:43.842770+0000","last_peered":"2024-02-14T17:17:43.842770+0000","last_clean":"2024-02-14T17:17:43.842770+0000","last_became_active":"2024-02-14T17:17:17.896118+0000","last_became_peered":"2024-02-14T17:17:17.896118+0000","last_unstale":"2024-02-14T17:17:43.842770+0000","last_undegraded":"2024-02-14T17:17:43.842770+0000","last_fullsized":"2024-02-14T17:17:43.842770+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:56:40.460111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845151+0000","last_change":"2024-02-14T17:17:21.926390+0000","last_active":"2024-02-14T17:17:43.845151+0000","last_peered":"2024-02-14T17:17:43.845151+0000","last_clean":"2024-02-14T17:17:43.845151+0000","last_became_active":"2024-02-14T17:17:21.925965+0000","last_became_peered":"2024-02-14T17:17:21.925965+0000","last_unstale":"2024-02-14T17:17:43.845151+0000","last_undegraded":"2024-02-14T17:17:43.845151+0000","last_fullsized":"2024-02-14T17:17:43.845151+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:11:58.289687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844298+0000","last_change":"2024-02-14T17:17:15.886283+0000","last_active":"2024-02-14T17:17:43.844298+0000","last_peered":"2024-02-14T17:17:43.844298+0000","last_clean":"2024-02-14T17:17:43.844298+0000","last_became_active":"2024-02-14T17:17:15.885808+0000","last_became_peered":"2024-02-14T17:17:15.885808+0000","last_unstale":"2024-02-14T17:17:43.844298+0000","last_undegraded":"2024-02-14T17:17:43.844298+0000","last_fullsized":"2024-02-14T17:17:43.844298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:45:14.411993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844264+0000","last_change":"2024-02-14T17:17:17.898169+0000","last_active":"2024-02-14T17:17:43.844264+0000","last_peered":"2024-02-14T17:17:43.844264+0000","last_clean":"2024-02-14T17:17:43.844264+0000","last_became_active":"2024-02-14T17:17:17.897377+0000","last_became_peered":"2024-02-14T17:17:17.897377+0000","last_unstale":"2024-02-14T17:17:43.844264+0000","last_undegraded":"2024-02-14T17:17:43.844264+0000","last_fullsized":"2024-02-14T17:17:43.844264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:23:58.178604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929739+0000","last_change":"2024-02-14T17:17:19.906734+0000","last_active":"2024-02-14T17:17:45.929739+0000","last_peered":"2024-02-14T17:17:45.929739+0000","last_clean":"2024-02-14T17:17:45.929739+0000","last_became_active":"2024-02-14T17:17:19.906024+0000","last_became_peered":"2024-02-14T17:17:19.906024+0000","last_unstale":"2024-02-14T17:17:45.929739+0000","last_undegraded":"2024-02-14T17:17:45.929739+0000","last_fullsized":"2024-02-14T17:17:45.929739+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:28:55.748001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843220+0000","last_change":"2024-02-14T17:17:21.921812+0000","last_active":"2024-02-14T17:17:43.843220+0000","last_peered":"2024-02-14T17:17:43.843220+0000","last_clean":"2024-02-14T17:17:43.843220+0000","last_became_active":"2024-02-14T17:17:21.921500+0000","last_became_peered":"2024-02-14T17:17:21.921500+0000","last_unstale":"2024-02-14T17:17:43.843220+0000","last_undegraded":"2024-02-14T17:17:43.843220+0000","last_fullsized":"2024-02-14T17:17:43.843220+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:48.154003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845252+0000","last_change":"2024-02-14T17:17:15.885041+0000","last_active":"2024-02-14T17:17:43.845252+0000","last_peered":"2024-02-14T17:17:43.845252+0000","last_clean":"2024-02-14T17:17:43.845252+0000","last_became_active":"2024-02-14T17:17:15.884070+0000","last_became_peered":"2024-02-14T17:17:15.884070+0000","last_unstale":"2024-02-14T17:17:43.845252+0000","last_undegraded":"2024-02-14T17:17:43.845252+0000","last_fullsized":"2024-02-14T17:17:43.845252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:17:02.064818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929610+0000","last_change":"2024-02-14T17:17:17.892591+0000","last_active":"2024-02-14T17:17:45.929610+0000","last_peered":"2024-02-14T17:17:45.929610+0000","last_clean":"2024-02-14T17:17:45.929610+0000","last_became_active":"2024-02-14T17:17:17.892127+0000","last_became_peered":"2024-02-14T17:17:17.892127+0000","last_unstale":"2024-02-14T17:17:45.929610+0000","last_undegraded":"2024-02-14T17:17:45.929610+0000","last_fullsized":"2024-02-14T17:17:45.929610+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:09:52.160369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843779+0000","last_change":"2024-02-14T17:17:19.906012+0000","last_active":"2024-02-14T17:17:43.843779+0000","last_peered":"2024-02-14T17:17:43.843779+0000","last_clean":"2024-02-14T17:17:43.843779+0000","last_became_active":"2024-02-14T17:17:19.905585+0000","last_became_peered":"2024-02-14T17:17:19.905585+0000","last_unstale":"2024-02-14T17:17:43.843779+0000","last_undegraded":"2024-02-14T17:17:43.843779+0000","last_fullsized":"2024-02-14T17:17:43.843779+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:46:39.095722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843121+0000","last_change":"2024-02-14T17:17:21.925826+0000","last_active":"2024-02-14T17:17:43.843121+0000","last_peered":"2024-02-14T17:17:43.843121+0000","last_clean":"2024-02-14T17:17:43.843121+0000","last_became_active":"2024-02-14T17:17:21.925438+0000","last_became_peered":"2024-02-14T17:17:21.925438+0000","last_unstale":"2024-02-14T17:17:43.843121+0000","last_undegraded":"2024-02-14T17:17:43.843121+0000","last_fullsized":"2024-02-14T17:17:43.843121+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:14:15.811566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843371+0000","last_change":"2024-02-14T17:17:15.884735+0000","last_active":"2024-02-14T17:17:43.843371+0000","last_peered":"2024-02-14T17:17:43.843371+0000","last_clean":"2024-02-14T17:17:43.843371+0000","last_became_active":"2024-02-14T17:17:15.884347+0000","last_became_peered":"2024-02-14T17:17:15.884347+0000","last_unstale":"2024-02-14T17:17:43.843371+0000","last_undegraded":"2024-02-14T17:17:43.843371+0000","last_fullsized":"2024-02-14T17:17:43.843371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:37:57.983734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843198+0000","last_change":"2024-02-14T17:17:17.893817+0000","last_active":"2024-02-14T17:17:43.843198+0000","last_peered":"2024-02-14T17:17:43.843198+0000","last_clean":"2024-02-14T17:17:43.843198+0000","last_became_active":"2024-02-14T17:17:17.892935+0000","last_became_peered":"2024-02-14T17:17:17.892935+0000","last_unstale":"2024-02-14T17:17:43.843198+0000","last_undegraded":"2024-02-14T17:17:43.843198+0000","last_fullsized":"2024-02-14T17:17:43.843198+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:07:50.191051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844939+0000","last_change":"2024-02-14T17:17:19.912172+0000","last_active":"2024-02-14T17:17:43.844939+0000","last_peered":"2024-02-14T17:17:43.844939+0000","last_clean":"2024-02-14T17:17:43.844939+0000","last_became_active":"2024-02-14T17:17:19.911528+0000","last_became_peered":"2024-02-14T17:17:19.911528+0000","last_unstale":"2024-02-14T17:17:43.844939+0000","last_undegraded":"2024-02-14T17:17:43.844939+0000","last_fullsized":"2024-02-14T17:17:43.844939+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:14:25.458332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845191+0000","last_change":"2024-02-14T17:17:21.926039+0000","last_active":"2024-02-14T17:17:43.845191+0000","last_peered":"2024-02-14T17:17:43.845191+0000","last_clean":"2024-02-14T17:17:43.845191+0000","last_became_active":"2024-02-14T17:17:21.925727+0000","last_became_peered":"2024-02-14T17:17:21.925727+0000","last_unstale":"2024-02-14T17:17:43.845191+0000","last_undegraded":"2024-02-14T17:17:43.845191+0000","last_fullsized":"2024-02-14T17:17:43.845191+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:34:38.358980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842771+0000","last_change":"2024-02-14T17:17:17.898095+0000","last_active":"2024-02-14T17:17:43.842771+0000","last_peered":"2024-02-14T17:17:43.842771+0000","last_clean":"2024-02-14T17:17:43.842771+0000","last_became_active":"2024-02-14T17:17:17.897231+0000","last_became_peered":"2024-02-14T17:17:17.897231+0000","last_unstale":"2024-02-14T17:17:43.842771+0000","last_undegraded":"2024-02-14T17:17:43.842771+0000","last_fullsized":"2024-02-14T17:17:43.842771+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:55:37.784640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841595+0000","last_change":"2024-02-14T17:17:15.884932+0000","last_active":"2024-02-14T17:17:43.841595+0000","last_peered":"2024-02-14T17:17:43.841595+0000","last_clean":"2024-02-14T17:17:43.841595+0000","last_became_active":"2024-02-14T17:17:15.884205+0000","last_became_peered":"2024-02-14T17:17:15.884205+0000","last_unstale":"2024-02-14T17:17:43.841595+0000","last_undegraded":"2024-02-14T17:17:43.841595+0000","last_fullsized":"2024-02-14T17:17:43.841595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:37:56.312318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846150+0000","last_change":"2024-02-14T17:17:19.912025+0000","last_active":"2024-02-14T17:17:43.846150+0000","last_peered":"2024-02-14T17:17:43.846150+0000","last_clean":"2024-02-14T17:17:43.846150+0000","last_became_active":"2024-02-14T17:17:19.911531+0000","last_became_peered":"2024-02-14T17:17:19.911531+0000","last_unstale":"2024-02-14T17:17:43.846150+0000","last_undegraded":"2024-02-14T17:17:43.846150+0000","last_fullsized":"2024-02-14T17:17:43.846150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:02:25.244444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843065+0000","last_change":"2024-02-14T17:17:21.921276+0000","last_active":"2024-02-14T17:17:43.843065+0000","last_peered":"2024-02-14T17:17:43.843065+0000","last_clean":"2024-02-14T17:17:43.843065+0000","last_became_active":"2024-02-14T17:17:21.920856+0000","last_became_peered":"2024-02-14T17:17:21.920856+0000","last_unstale":"2024-02-14T17:17:43.843065+0000","last_undegraded":"2024-02-14T17:17:43.843065+0000","last_fullsized":"2024-02-14T17:17:43.843065+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:04:49.768478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842758+0000","last_change":"2024-02-14T17:17:17.895183+0000","last_active":"2024-02-14T17:17:43.842758+0000","last_peered":"2024-02-14T17:17:43.842758+0000","last_clean":"2024-02-14T17:17:43.842758+0000","last_became_active":"2024-02-14T17:17:17.894667+0000","last_became_peered":"2024-02-14T17:17:17.894667+0000","last_unstale":"2024-02-14T17:17:43.842758+0000","last_undegraded":"2024-02-14T17:17:43.842758+0000","last_fullsized":"2024-02-14T17:17:43.842758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:10:49.864207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842981+0000","last_change":"2024-02-14T17:17:15.884967+0000","last_active":"2024-02-14T17:17:43.842981+0000","last_peered":"2024-02-14T17:17:43.842981+0000","last_clean":"2024-02-14T17:17:43.842981+0000","last_became_active":"2024-02-14T17:17:15.883693+0000","last_became_peered":"2024-02-14T17:17:15.883693+0000","last_unstale":"2024-02-14T17:17:43.842981+0000","last_undegraded":"2024-02-14T17:17:43.842981+0000","last_fullsized":"2024-02-14T17:17:43.842981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:56:34.889112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.939941+0000","last_change":"2024-02-14T17:17:01.905589+0000","last_active":"2024-02-14T17:17:45.939941+0000","last_peered":"2024-02-14T17:17:45.939941+0000","last_clean":"2024-02-14T17:17:45.939941+0000","last_became_active":"2024-02-14T17:17:01.905307+0000","last_became_peered":"2024-02-14T17:17:01.905307+0000","last_unstale":"2024-02-14T17:17:45.939941+0000","last_undegraded":"2024-02-14T17:17:45.939941+0000","last_fullsized":"2024-02-14T17:17:45.939941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:00.892872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:00.892872+0000","last_clean_scrub_stamp":"2024-02-14T17:17:00.892872+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:14:45.527406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844466+0000","last_change":"2024-02-14T17:17:19.909415+0000","last_active":"2024-02-14T17:17:43.844466+0000","last_peered":"2024-02-14T17:17:43.844466+0000","last_clean":"2024-02-14T17:17:43.844466+0000","last_became_active":"2024-02-14T17:17:19.908694+0000","last_became_peered":"2024-02-14T17:17:19.908694+0000","last_unstale":"2024-02-14T17:17:43.844466+0000","last_undegraded":"2024-02-14T17:17:43.844466+0000","last_fullsized":"2024-02-14T17:17:43.844466+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:25:11.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844537+0000","last_change":"2024-02-14T17:17:21.924435+0000","last_active":"2024-02-14T17:17:43.844537+0000","last_peered":"2024-02-14T17:17:43.844537+0000","last_clean":"2024-02-14T17:17:43.844537+0000","last_became_active":"2024-02-14T17:17:21.922537+0000","last_became_peered":"2024-02-14T17:17:21.922537+0000","last_unstale":"2024-02-14T17:17:43.844537+0000","last_undegraded":"2024-02-14T17:17:43.844537+0000","last_fullsized":"2024-02-14T17:17:43.844537+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:08:46.968206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841920+0000","last_change":"2024-02-14T17:17:17.894386+0000","last_active":"2024-02-14T17:17:43.841920+0000","last_peered":"2024-02-14T17:17:43.841920+0000","last_clean":"2024-02-14T17:17:43.841920+0000","last_became_active":"2024-02-14T17:17:17.894022+0000","last_became_peered":"2024-02-14T17:17:17.894022+0000","last_unstale":"2024-02-14T17:17:43.841920+0000","last_undegraded":"2024-02-14T17:17:43.841920+0000","last_fullsized":"2024-02-14T17:17:43.841920+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:46:42.476920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841892+0000","last_change":"2024-02-14T17:17:15.887911+0000","last_active":"2024-02-14T17:17:43.841892+0000","last_peered":"2024-02-14T17:17:43.841892+0000","last_clean":"2024-02-14T17:17:43.841892+0000","last_became_active":"2024-02-14T17:17:15.887051+0000","last_became_peered":"2024-02-14T17:17:15.887051+0000","last_unstale":"2024-02-14T17:17:43.841892+0000","last_undegraded":"2024-02-14T17:17:43.841892+0000","last_fullsized":"2024-02-14T17:17:43.841892+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:21:32.811193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843507+0000","last_change":"2024-02-14T17:17:19.906830+0000","last_active":"2024-02-14T17:17:43.843507+0000","last_peered":"2024-02-14T17:17:43.843507+0000","last_clean":"2024-02-14T17:17:43.843507+0000","last_became_active":"2024-02-14T17:17:19.906113+0000","last_became_peered":"2024-02-14T17:17:19.906113+0000","last_unstale":"2024-02-14T17:17:43.843507+0000","last_undegraded":"2024-02-14T17:17:43.843507+0000","last_fullsized":"2024-02-14T17:17:43.843507+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:28:01.451955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843127+0000","last_change":"2024-02-14T17:17:21.920517+0000","last_active":"2024-02-14T17:17:43.843127+0000","last_peered":"2024-02-14T17:17:43.843127+0000","last_clean":"2024-02-14T17:17:43.843127+0000","last_became_active":"2024-02-14T17:17:21.920126+0000","last_became_peered":"2024-02-14T17:17:21.920126+0000","last_unstale":"2024-02-14T17:17:43.843127+0000","last_undegraded":"2024-02-14T17:17:43.843127+0000","last_fullsized":"2024-02-14T17:17:43.843127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:11:16.287000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845285+0000","last_change":"2024-02-14T17:17:15.889334+0000","last_active":"2024-02-14T17:17:43.845285+0000","last_peered":"2024-02-14T17:17:43.845285+0000","last_clean":"2024-02-14T17:17:43.845285+0000","last_became_active":"2024-02-14T17:17:15.888605+0000","last_became_peered":"2024-02-14T17:17:15.888605+0000","last_unstale":"2024-02-14T17:17:43.845285+0000","last_undegraded":"2024-02-14T17:17:43.845285+0000","last_fullsized":"2024-02-14T17:17:43.845285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:45:08.389257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929523+0000","last_change":"2024-02-14T17:17:17.894539+0000","last_active":"2024-02-14T17:17:45.929523+0000","last_peered":"2024-02-14T17:17:45.929523+0000","last_clean":"2024-02-14T17:17:45.929523+0000","last_became_active":"2024-02-14T17:17:17.894127+0000","last_became_peered":"2024-02-14T17:17:17.894127+0000","last_unstale":"2024-02-14T17:17:45.929523+0000","last_undegraded":"2024-02-14T17:17:45.929523+0000","last_fullsized":"2024-02-14T17:17:45.929523+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:18:06.678417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929730+0000","last_change":"2024-02-14T17:17:19.906361+0000","last_active":"2024-02-14T17:17:45.929730+0000","last_peered":"2024-02-14T17:17:45.929730+0000","last_clean":"2024-02-14T17:17:45.929730+0000","last_became_active":"2024-02-14T17:17:19.906018+0000","last_became_peered":"2024-02-14T17:17:19.906018+0000","last_unstale":"2024-02-14T17:17:45.929730+0000","last_undegraded":"2024-02-14T17:17:45.929730+0000","last_fullsized":"2024-02-14T17:17:45.929730+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:24:52.761717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841933+0000","last_change":"2024-02-14T17:17:21.921770+0000","last_active":"2024-02-14T17:17:43.841933+0000","last_peered":"2024-02-14T17:17:43.841933+0000","last_clean":"2024-02-14T17:17:43.841933+0000","last_became_active":"2024-02-14T17:17:21.921354+0000","last_became_peered":"2024-02-14T17:17:21.921354+0000","last_unstale":"2024-02-14T17:17:43.841933+0000","last_undegraded":"2024-02-14T17:17:43.841933+0000","last_fullsized":"2024-02-14T17:17:43.841933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:01:22.458852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845089+0000","last_change":"2024-02-14T17:17:15.885290+0000","last_active":"2024-02-14T17:17:43.845089+0000","last_peered":"2024-02-14T17:17:43.845089+0000","last_clean":"2024-02-14T17:17:43.845089+0000","last_became_active":"2024-02-14T17:17:15.885055+0000","last_became_peered":"2024-02-14T17:17:15.885055+0000","last_unstale":"2024-02-14T17:17:43.845089+0000","last_undegraded":"2024-02-14T17:17:43.845089+0000","last_fullsized":"2024-02-14T17:17:43.845089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:10:09.199166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846659+0000","last_change":"2024-02-14T17:17:17.897066+0000","last_active":"2024-02-14T17:17:43.846659+0000","last_peered":"2024-02-14T17:17:43.846659+0000","last_clean":"2024-02-14T17:17:43.846659+0000","last_became_active":"2024-02-14T17:17:17.896633+0000","last_became_peered":"2024-02-14T17:17:17.896633+0000","last_unstale":"2024-02-14T17:17:43.846659+0000","last_undegraded":"2024-02-14T17:17:43.846659+0000","last_fullsized":"2024-02-14T17:17:43.846659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:42:12.404731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929514+0000","last_change":"2024-02-14T17:17:19.911895+0000","last_active":"2024-02-14T17:17:45.929514+0000","last_peered":"2024-02-14T17:17:45.929514+0000","last_clean":"2024-02-14T17:17:45.929514+0000","last_became_active":"2024-02-14T17:17:19.911225+0000","last_became_peered":"2024-02-14T17:17:19.911225+0000","last_unstale":"2024-02-14T17:17:45.929514+0000","last_undegraded":"2024-02-14T17:17:45.929514+0000","last_fullsized":"2024-02-14T17:17:45.929514+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:00.612864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844197+0000","last_change":"2024-02-14T17:17:21.926560+0000","last_active":"2024-02-14T17:17:43.844197+0000","last_peered":"2024-02-14T17:17:43.844197+0000","last_clean":"2024-02-14T17:17:43.844197+0000","last_became_active":"2024-02-14T17:17:21.926284+0000","last_became_peered":"2024-02-14T17:17:21.926284+0000","last_unstale":"2024-02-14T17:17:43.844197+0000","last_undegraded":"2024-02-14T17:17:43.844197+0000","last_fullsized":"2024-02-14T17:17:43.844197+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:46.148924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845128+0000","last_change":"2024-02-14T17:17:15.886443+0000","last_active":"2024-02-14T17:17:43.845128+0000","last_peered":"2024-02-14T17:17:43.845128+0000","last_clean":"2024-02-14T17:17:43.845128+0000","last_became_active":"2024-02-14T17:17:15.886060+0000","last_became_peered":"2024-02-14T17:17:15.886060+0000","last_unstale":"2024-02-14T17:17:43.845128+0000","last_undegraded":"2024-02-14T17:17:43.845128+0000","last_fullsized":"2024-02-14T17:17:43.845128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:22:30.648925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844531+0000","last_change":"2024-02-14T17:17:17.894744+0000","last_active":"2024-02-14T17:17:43.844531+0000","last_peered":"2024-02-14T17:17:43.844531+0000","last_clean":"2024-02-14T17:17:43.844531+0000","last_became_active":"2024-02-14T17:17:17.894442+0000","last_became_peered":"2024-02-14T17:17:17.894442+0000","last_unstale":"2024-02-14T17:17:43.844531+0000","last_undegraded":"2024-02-14T17:17:43.844531+0000","last_fullsized":"2024-02-14T17:17:43.844531+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:02.060037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842586+0000","last_change":"2024-02-14T17:17:19.909156+0000","last_active":"2024-02-14T17:17:43.842586+0000","last_peered":"2024-02-14T17:17:43.842586+0000","last_clean":"2024-02-14T17:17:43.842586+0000","last_became_active":"2024-02-14T17:17:19.908967+0000","last_became_peered":"2024-02-14T17:17:19.908967+0000","last_unstale":"2024-02-14T17:17:43.842586+0000","last_undegraded":"2024-02-14T17:17:43.842586+0000","last_fullsized":"2024-02-14T17:17:43.842586+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:50:36.076268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844937+0000","last_change":"2024-02-14T17:17:21.924320+0000","last_active":"2024-02-14T17:17:43.844937+0000","last_peered":"2024-02-14T17:17:43.844937+0000","last_clean":"2024-02-14T17:17:43.844937+0000","last_became_active":"2024-02-14T17:17:21.922224+0000","last_became_peered":"2024-02-14T17:17:21.922224+0000","last_unstale":"2024-02-14T17:17:43.844937+0000","last_undegraded":"2024-02-14T17:17:43.844937+0000","last_fullsized":"2024-02-14T17:17:43.844937+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:44:30.248401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929572+0000","last_change":"2024-02-14T17:17:19.909427+0000","last_active":"2024-02-14T17:17:45.929572+0000","last_peered":"2024-02-14T17:17:45.929572+0000","last_clean":"2024-02-14T17:17:45.929572+0000","last_became_active":"2024-02-14T17:17:19.909084+0000","last_became_peered":"2024-02-14T17:17:19.909084+0000","last_unstale":"2024-02-14T17:17:45.929572+0000","last_undegraded":"2024-02-14T17:17:45.929572+0000","last_fullsized":"2024-02-14T17:17:45.929572+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:16.883480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842121+0000","last_change":"2024-02-14T17:17:15.884678+0000","last_active":"2024-02-14T17:17:43.842121+0000","last_peered":"2024-02-14T17:17:43.842121+0000","last_clean":"2024-02-14T17:17:43.842121+0000","last_became_active":"2024-02-14T17:17:15.883977+0000","last_became_peered":"2024-02-14T17:17:15.883977+0000","last_unstale":"2024-02-14T17:17:43.842121+0000","last_undegraded":"2024-02-14T17:17:43.842121+0000","last_fullsized":"2024-02-14T17:17:43.842121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:10:56.213069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842956+0000","last_change":"2024-02-14T17:17:17.897213+0000","last_active":"2024-02-14T17:17:43.842956+0000","last_peered":"2024-02-14T17:17:43.842956+0000","last_clean":"2024-02-14T17:17:43.842956+0000","last_became_active":"2024-02-14T17:17:17.896228+0000","last_became_peered":"2024-02-14T17:17:17.896228+0000","last_unstale":"2024-02-14T17:17:43.842956+0000","last_undegraded":"2024-02-14T17:17:43.842956+0000","last_fullsized":"2024-02-14T17:17:43.842956+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:07:27.384481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843608+0000","last_change":"2024-02-14T17:17:21.925909+0000","last_active":"2024-02-14T17:17:43.843608+0000","last_peered":"2024-02-14T17:17:43.843608+0000","last_clean":"2024-02-14T17:17:43.843608+0000","last_became_active":"2024-02-14T17:17:21.925586+0000","last_became_peered":"2024-02-14T17:17:21.925586+0000","last_unstale":"2024-02-14T17:17:43.843608+0000","last_undegraded":"2024-02-14T17:17:43.843608+0000","last_fullsized":"2024-02-14T17:17:43.843608+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:41:21.303093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845849+0000","last_change":"2024-02-14T17:17:19.911990+0000","last_active":"2024-02-14T17:17:43.845849+0000","last_peered":"2024-02-14T17:17:43.845849+0000","last_clean":"2024-02-14T17:17:43.845849+0000","last_became_active":"2024-02-14T17:17:19.911486+0000","last_became_peered":"2024-02-14T17:17:19.911486+0000","last_unstale":"2024-02-14T17:17:43.845849+0000","last_undegraded":"2024-02-14T17:17:43.845849+0000","last_fullsized":"2024-02-14T17:17:43.845849+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:32:59.643328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841749+0000","last_change":"2024-02-14T17:17:15.885145+0000","last_active":"2024-02-14T17:17:43.841749+0000","last_peered":"2024-02-14T17:17:43.841749+0000","last_clean":"2024-02-14T17:17:43.841749+0000","last_became_active":"2024-02-14T17:17:15.884189+0000","last_became_peered":"2024-02-14T17:17:15.884189+0000","last_unstale":"2024-02-14T17:17:43.841749+0000","last_undegraded":"2024-02-14T17:17:43.841749+0000","last_fullsized":"2024-02-14T17:17:43.841749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:10:48.334334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843264+0000","last_change":"2024-02-14T17:17:17.897930+0000","last_active":"2024-02-14T17:17:43.843264+0000","last_peered":"2024-02-14T17:17:43.843264+0000","last_clean":"2024-02-14T17:17:43.843264+0000","last_became_active":"2024-02-14T17:17:17.896426+0000","last_became_peered":"2024-02-14T17:17:17.896426+0000","last_unstale":"2024-02-14T17:17:43.843264+0000","last_undegraded":"2024-02-14T17:17:43.843264+0000","last_fullsized":"2024-02-14T17:17:43.843264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:28:39.643480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842993+0000","last_change":"2024-02-14T17:17:21.926041+0000","last_active":"2024-02-14T17:17:43.842993+0000","last_peered":"2024-02-14T17:17:43.842993+0000","last_clean":"2024-02-14T17:17:43.842993+0000","last_became_active":"2024-02-14T17:17:21.925745+0000","last_became_peered":"2024-02-14T17:17:21.925745+0000","last_unstale":"2024-02-14T17:17:43.842993+0000","last_undegraded":"2024-02-14T17:17:43.842993+0000","last_fullsized":"2024-02-14T17:17:43.842993+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:30:10.719144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844328+0000","last_change":"2024-02-14T17:17:15.886510+0000","last_active":"2024-02-14T17:17:43.844328+0000","last_peered":"2024-02-14T17:17:43.844328+0000","last_clean":"2024-02-14T17:17:43.844328+0000","last_became_active":"2024-02-14T17:17:15.885806+0000","last_became_peered":"2024-02-14T17:17:15.885806+0000","last_unstale":"2024-02-14T17:17:43.844328+0000","last_undegraded":"2024-02-14T17:17:43.844328+0000","last_fullsized":"2024-02-14T17:17:43.844328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:25:22.081089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843675+0000","last_change":"2024-02-14T17:17:17.894716+0000","last_active":"2024-02-14T17:17:43.843675+0000","last_peered":"2024-02-14T17:17:43.843675+0000","last_clean":"2024-02-14T17:17:43.843675+0000","last_became_active":"2024-02-14T17:17:17.894342+0000","last_became_peered":"2024-02-14T17:17:17.894342+0000","last_unstale":"2024-02-14T17:17:43.843675+0000","last_undegraded":"2024-02-14T17:17:43.843675+0000","last_fullsized":"2024-02-14T17:17:43.843675+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:56:10.807313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844010+0000","last_change":"2024-02-14T17:17:19.906769+0000","last_active":"2024-02-14T17:17:43.844010+0000","last_peered":"2024-02-14T17:17:43.844010+0000","last_clean":"2024-02-14T17:17:43.844010+0000","last_became_active":"2024-02-14T17:17:19.906418+0000","last_became_peered":"2024-02-14T17:17:19.906418+0000","last_unstale":"2024-02-14T17:17:43.844010+0000","last_undegraded":"2024-02-14T17:17:43.844010+0000","last_fullsized":"2024-02-14T17:17:43.844010+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:00:05.015696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844734+0000","last_change":"2024-02-14T17:17:21.926425+0000","last_active":"2024-02-14T17:17:43.844734+0000","last_peered":"2024-02-14T17:17:43.844734+0000","last_clean":"2024-02-14T17:17:43.844734+0000","last_became_active":"2024-02-14T17:17:21.926124+0000","last_became_peered":"2024-02-14T17:17:21.926124+0000","last_unstale":"2024-02-14T17:17:43.844734+0000","last_undegraded":"2024-02-14T17:17:43.844734+0000","last_fullsized":"2024-02-14T17:17:43.844734+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:39:01.307445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929556+0000","last_change":"2024-02-14T17:17:19.909181+0000","last_active":"2024-02-14T17:17:45.929556+0000","last_peered":"2024-02-14T17:17:45.929556+0000","last_clean":"2024-02-14T17:17:45.929556+0000","last_became_active":"2024-02-14T17:17:19.908691+0000","last_became_peered":"2024-02-14T17:17:19.908691+0000","last_unstale":"2024-02-14T17:17:45.929556+0000","last_undegraded":"2024-02-14T17:17:45.929556+0000","last_fullsized":"2024-02-14T17:17:45.929556+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:48:42.201337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845247+0000","last_change":"2024-02-14T17:17:15.887897+0000","last_active":"2024-02-14T17:17:43.845247+0000","last_peered":"2024-02-14T17:17:43.845247+0000","last_clean":"2024-02-14T17:17:43.845247+0000","last_became_active":"2024-02-14T17:17:15.887606+0000","last_became_peered":"2024-02-14T17:17:15.887606+0000","last_unstale":"2024-02-14T17:17:43.845247+0000","last_undegraded":"2024-02-14T17:17:43.845247+0000","last_fullsized":"2024-02-14T17:17:43.845247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:23.196204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843505+0000","last_change":"2024-02-14T17:17:17.892849+0000","last_active":"2024-02-14T17:17:43.843505+0000","last_peered":"2024-02-14T17:17:43.843505+0000","last_clean":"2024-02-14T17:17:43.843505+0000","last_became_active":"2024-02-14T17:17:17.891904+0000","last_became_peered":"2024-02-14T17:17:17.891904+0000","last_unstale":"2024-02-14T17:17:43.843505+0000","last_undegraded":"2024-02-14T17:17:43.843505+0000","last_fullsized":"2024-02-14T17:17:43.843505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:16:56.264422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843039+0000","last_change":"2024-02-14T17:17:21.926248+0000","last_active":"2024-02-14T17:17:43.843039+0000","last_peered":"2024-02-14T17:17:43.843039+0000","last_clean":"2024-02-14T17:17:43.843039+0000","last_became_active":"2024-02-14T17:17:21.925848+0000","last_became_peered":"2024-02-14T17:17:21.925848+0000","last_unstale":"2024-02-14T17:17:43.843039+0000","last_undegraded":"2024-02-14T17:17:43.843039+0000","last_fullsized":"2024-02-14T17:17:43.843039+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:23:37.998457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845507+0000","last_change":"2024-02-14T17:17:15.886505+0000","last_active":"2024-02-14T17:17:43.845507+0000","last_peered":"2024-02-14T17:17:43.845507+0000","last_clean":"2024-02-14T17:17:43.845507+0000","last_became_active":"2024-02-14T17:17:15.886136+0000","last_became_peered":"2024-02-14T17:17:15.886136+0000","last_unstale":"2024-02-14T17:17:43.845507+0000","last_undegraded":"2024-02-14T17:17:43.845507+0000","last_fullsized":"2024-02-14T17:17:43.845507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:03:52.304724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844959+0000","last_change":"2024-02-14T17:17:17.893380+0000","last_active":"2024-02-14T17:17:43.844959+0000","last_peered":"2024-02-14T17:17:43.844959+0000","last_clean":"2024-02-14T17:17:43.844959+0000","last_became_active":"2024-02-14T17:17:17.892442+0000","last_became_peered":"2024-02-14T17:17:17.892442+0000","last_unstale":"2024-02-14T17:17:43.844959+0000","last_undegraded":"2024-02-14T17:17:43.844959+0000","last_fullsized":"2024-02-14T17:17:43.844959+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:14:17.579346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842978+0000","last_change":"2024-02-14T17:17:19.906774+0000","last_active":"2024-02-14T17:17:43.842978+0000","last_peered":"2024-02-14T17:17:43.842978+0000","last_clean":"2024-02-14T17:17:43.842978+0000","last_became_active":"2024-02-14T17:17:19.906391+0000","last_became_peered":"2024-02-14T17:17:19.906391+0000","last_unstale":"2024-02-14T17:17:43.842978+0000","last_undegraded":"2024-02-14T17:17:43.842978+0000","last_fullsized":"2024-02-14T17:17:43.842978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:27:37.483011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841711+0000","last_change":"2024-02-14T17:17:21.926162+0000","last_active":"2024-02-14T17:17:43.841711+0000","last_peered":"2024-02-14T17:17:43.841711+0000","last_clean":"2024-02-14T17:17:43.841711+0000","last_became_active":"2024-02-14T17:17:21.925850+0000","last_became_peered":"2024-02-14T17:17:21.925850+0000","last_unstale":"2024-02-14T17:17:43.841711+0000","last_undegraded":"2024-02-14T17:17:43.841711+0000","last_fullsized":"2024-02-14T17:17:43.841711+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:10.309431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843783+0000","last_change":"2024-02-14T17:17:15.889369+0000","last_active":"2024-02-14T17:17:43.843783+0000","last_peered":"2024-02-14T17:17:43.843783+0000","last_clean":"2024-02-14T17:17:43.843783+0000","last_became_active":"2024-02-14T17:17:15.888846+0000","last_became_peered":"2024-02-14T17:17:15.888846+0000","last_unstale":"2024-02-14T17:17:43.843783+0000","last_undegraded":"2024-02-14T17:17:43.843783+0000","last_fullsized":"2024-02-14T17:17:43.843783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:44:26.860854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844275+0000","last_change":"2024-02-14T17:17:17.893073+0000","last_active":"2024-02-14T17:17:43.844275+0000","last_peered":"2024-02-14T17:17:43.844275+0000","last_clean":"2024-02-14T17:17:43.844275+0000","last_became_active":"2024-02-14T17:17:17.892792+0000","last_became_peered":"2024-02-14T17:17:17.892792+0000","last_unstale":"2024-02-14T17:17:43.844275+0000","last_undegraded":"2024-02-14T17:17:43.844275+0000","last_fullsized":"2024-02-14T17:17:43.844275+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:06:53.719923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843367+0000","last_change":"2024-02-14T17:17:19.910303+0000","last_active":"2024-02-14T17:17:43.843367+0000","last_peered":"2024-02-14T17:17:43.843367+0000","last_clean":"2024-02-14T17:17:43.843367+0000","last_became_active":"2024-02-14T17:17:19.909941+0000","last_became_peered":"2024-02-14T17:17:19.909941+0000","last_unstale":"2024-02-14T17:17:43.843367+0000","last_undegraded":"2024-02-14T17:17:43.843367+0000","last_fullsized":"2024-02-14T17:17:43.843367+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:23:13.437631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842064+0000","last_change":"2024-02-14T17:17:21.920575+0000","last_active":"2024-02-14T17:17:43.842064+0000","last_peered":"2024-02-14T17:17:43.842064+0000","last_clean":"2024-02-14T17:17:43.842064+0000","last_became_active":"2024-02-14T17:17:21.920102+0000","last_became_peered":"2024-02-14T17:17:21.920102+0000","last_unstale":"2024-02-14T17:17:43.842064+0000","last_undegraded":"2024-02-14T17:17:43.842064+0000","last_fullsized":"2024-02-14T17:17:43.842064+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:38:08.152329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844786+0000","last_change":"2024-02-14T17:17:15.884968+0000","last_active":"2024-02-14T17:17:43.844786+0000","last_peered":"2024-02-14T17:17:43.844786+0000","last_clean":"2024-02-14T17:17:43.844786+0000","last_became_active":"2024-02-14T17:17:15.884065+0000","last_became_peered":"2024-02-14T17:17:15.884065+0000","last_unstale":"2024-02-14T17:17:43.844786+0000","last_undegraded":"2024-02-14T17:17:43.844786+0000","last_fullsized":"2024-02-14T17:17:43.844786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:17:31.935542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845206+0000","last_change":"2024-02-14T17:17:17.895683+0000","last_active":"2024-02-14T17:17:43.845206+0000","last_peered":"2024-02-14T17:17:43.845206+0000","last_clean":"2024-02-14T17:17:43.845206+0000","last_became_active":"2024-02-14T17:17:17.895289+0000","last_became_peered":"2024-02-14T17:17:17.895289+0000","last_unstale":"2024-02-14T17:17:43.845206+0000","last_undegraded":"2024-02-14T17:17:43.845206+0000","last_fullsized":"2024-02-14T17:17:43.845206+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:42:18.675102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844023+0000","last_change":"2024-02-14T17:17:19.907060+0000","last_active":"2024-02-14T17:17:43.844023+0000","last_peered":"2024-02-14T17:17:43.844023+0000","last_clean":"2024-02-14T17:17:43.844023+0000","last_became_active":"2024-02-14T17:17:19.906748+0000","last_became_peered":"2024-02-14T17:17:19.906748+0000","last_unstale":"2024-02-14T17:17:43.844023+0000","last_undegraded":"2024-02-14T17:17:43.844023+0000","last_fullsized":"2024-02-14T17:17:43.844023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:21:40.555140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738392,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 17:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.438},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T17:17:59.356 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-14T17:17:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:17:59.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:59 smithi082 ceph-mon[25796]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-14T17:17:59.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:59 smithi082 ceph-mon[25796]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:17:59.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:17:59 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:18:00.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:59 smithi120 ceph-mon[30899]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-14T17:18:00.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:59 smithi120 ceph-mon[30899]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:18:00.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:17:59 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:18:00.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:59 smithi091 ceph-mon[30290]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-14T17:18:00.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:59 smithi091 ceph-mon[30290]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:18:00.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:17:59 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:18:01.782 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:18:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-02-14T17:18:01.984 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:01 smithi082 ceph-mon[25796]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-14T17:18:02.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:01 smithi120 ceph-mon[30899]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-14T17:18:02.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:01 smithi091 ceph-mon[30290]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-14T17:18:02.468 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-02-14T17:17:59.855562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96612,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126556,"statfs":{"total":287972524032,"available":287873593344,"internally_reserved":0,"allocated":3158016,"data_stored":2100345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003179"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842634+0000","last_change":"2024-02-14T17:17:21.921850+0000","last_active":"2024-02-14T17:17:43.842634+0000","last_peered":"2024-02-14T17:17:43.842634+0000","last_clean":"2024-02-14T17:17:43.842634+0000","last_became_active":"2024-02-14T17:17:21.921386+0000","last_became_peered":"2024-02-14T17:17:21.921386+0000","last_unstale":"2024-02-14T17:17:43.842634+0000","last_undegraded":"2024-02-14T17:17:43.842634+0000","last_fullsized":"2024-02-14T17:17:43.842634+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:15:41.278032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842481+0000","last_change":"2024-02-14T17:17:19.905875+0000","last_active":"2024-02-14T17:17:43.842481+0000","last_peered":"2024-02-14T17:17:43.842481+0000","last_clean":"2024-02-14T17:17:43.842481+0000","last_became_active":"2024-02-14T17:17:19.905480+0000","last_became_peered":"2024-02-14T17:17:19.905480+0000","last_unstale":"2024-02-14T17:17:43.842481+0000","last_undegraded":"2024-02-14T17:17:43.842481+0000","last_fullsized":"2024-02-14T17:17:43.842481+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:28:14.250316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846463+0000","last_change":"2024-02-14T17:17:15.884394+0000","last_active":"2024-02-14T17:17:43.846463+0000","last_peered":"2024-02-14T17:17:43.846463+0000","last_clean":"2024-02-14T17:17:43.846463+0000","last_became_active":"2024-02-14T17:17:15.883878+0000","last_became_peered":"2024-02-14T17:17:15.883878+0000","last_unstale":"2024-02-14T17:17:43.846463+0000","last_undegraded":"2024-02-14T17:17:43.846463+0000","last_fullsized":"2024-02-14T17:17:43.846463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:49:21.806559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844657+0000","last_change":"2024-02-14T17:17:17.894753+0000","last_active":"2024-02-14T17:17:43.844657+0000","last_peered":"2024-02-14T17:17:43.844657+0000","last_clean":"2024-02-14T17:17:43.844657+0000","last_became_active":"2024-02-14T17:17:17.894487+0000","last_became_peered":"2024-02-14T17:17:17.894487+0000","last_unstale":"2024-02-14T17:17:43.844657+0000","last_undegraded":"2024-02-14T17:17:43.844657+0000","last_fullsized":"2024-02-14T17:17:43.844657+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:20:48.068097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841649+0000","last_change":"2024-02-14T17:17:21.919780+0000","last_active":"2024-02-14T17:17:43.841649+0000","last_peered":"2024-02-14T17:17:43.841649+0000","last_clean":"2024-02-14T17:17:43.841649+0000","last_became_active":"2024-02-14T17:17:21.919402+0000","last_became_peered":"2024-02-14T17:17:21.919402+0000","last_unstale":"2024-02-14T17:17:43.841649+0000","last_undegraded":"2024-02-14T17:17:43.841649+0000","last_fullsized":"2024-02-14T17:17:43.841649+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:58:10.934431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842327+0000","last_change":"2024-02-14T17:17:19.910232+0000","last_active":"2024-02-14T17:17:43.842327+0000","last_peered":"2024-02-14T17:17:43.842327+0000","last_clean":"2024-02-14T17:17:43.842327+0000","last_became_active":"2024-02-14T17:17:19.909926+0000","last_became_peered":"2024-02-14T17:17:19.909926+0000","last_unstale":"2024-02-14T17:17:43.842327+0000","last_undegraded":"2024-02-14T17:17:43.842327+0000","last_fullsized":"2024-02-14T17:17:43.842327+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:46:43.286543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844931+0000","last_change":"2024-02-14T17:17:15.883865+0000","last_active":"2024-02-14T17:17:43.844931+0000","last_peered":"2024-02-14T17:17:43.844931+0000","last_clean":"2024-02-14T17:17:43.844931+0000","last_became_active":"2024-02-14T17:17:15.883419+0000","last_became_peered":"2024-02-14T17:17:15.883419+0000","last_unstale":"2024-02-14T17:17:43.844931+0000","last_undegraded":"2024-02-14T17:17:43.844931+0000","last_fullsized":"2024-02-14T17:17:43.844931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:36:02.857579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846034+0000","last_change":"2024-02-14T17:17:17.895152+0000","last_active":"2024-02-14T17:17:43.846034+0000","last_peered":"2024-02-14T17:17:43.846034+0000","last_clean":"2024-02-14T17:17:43.846034+0000","last_became_active":"2024-02-14T17:17:17.894425+0000","last_became_peered":"2024-02-14T17:17:17.894425+0000","last_unstale":"2024-02-14T17:17:43.846034+0000","last_undegraded":"2024-02-14T17:17:43.846034+0000","last_fullsized":"2024-02-14T17:17:43.846034+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:48:38.940957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843540+0000","last_change":"2024-02-14T17:17:21.920558+0000","last_active":"2024-02-14T17:17:43.843540+0000","last_peered":"2024-02-14T17:17:43.843540+0000","last_clean":"2024-02-14T17:17:43.843540+0000","last_became_active":"2024-02-14T17:17:21.920280+0000","last_became_peered":"2024-02-14T17:17:21.920280+0000","last_unstale":"2024-02-14T17:17:43.843540+0000","last_undegraded":"2024-02-14T17:17:43.843540+0000","last_fullsized":"2024-02-14T17:17:43.843540+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:50:52.929464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842813+0000","last_change":"2024-02-14T17:17:19.909807+0000","last_active":"2024-02-14T17:17:43.842813+0000","last_peered":"2024-02-14T17:17:43.842813+0000","last_clean":"2024-02-14T17:17:43.842813+0000","last_became_active":"2024-02-14T17:17:19.908926+0000","last_became_peered":"2024-02-14T17:17:19.908926+0000","last_unstale":"2024-02-14T17:17:43.842813+0000","last_undegraded":"2024-02-14T17:17:43.842813+0000","last_fullsized":"2024-02-14T17:17:43.842813+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:02:31.200331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845398+0000","last_change":"2024-02-14T17:17:15.888439+0000","last_active":"2024-02-14T17:17:43.845398+0000","last_peered":"2024-02-14T17:17:43.845398+0000","last_clean":"2024-02-14T17:17:43.845398+0000","last_became_active":"2024-02-14T17:17:15.888083+0000","last_became_peered":"2024-02-14T17:17:15.888083+0000","last_unstale":"2024-02-14T17:17:43.845398+0000","last_undegraded":"2024-02-14T17:17:43.845398+0000","last_fullsized":"2024-02-14T17:17:43.845398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:10:33.711513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844400+0000","last_change":"2024-02-14T17:17:17.892428+0000","last_active":"2024-02-14T17:17:43.844400+0000","last_peered":"2024-02-14T17:17:43.844400+0000","last_clean":"2024-02-14T17:17:43.844400+0000","last_became_active":"2024-02-14T17:17:17.891911+0000","last_became_peered":"2024-02-14T17:17:17.891911+0000","last_unstale":"2024-02-14T17:17:43.844400+0000","last_undegraded":"2024-02-14T17:17:43.844400+0000","last_fullsized":"2024-02-14T17:17:43.844400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:26:03.087984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842722+0000","last_change":"2024-02-14T17:17:21.922077+0000","last_active":"2024-02-14T17:17:43.842722+0000","last_peered":"2024-02-14T17:17:43.842722+0000","last_clean":"2024-02-14T17:17:43.842722+0000","last_became_active":"2024-02-14T17:17:21.921509+0000","last_became_peered":"2024-02-14T17:17:21.921509+0000","last_unstale":"2024-02-14T17:17:43.842722+0000","last_undegraded":"2024-02-14T17:17:43.842722+0000","last_fullsized":"2024-02-14T17:17:43.842722+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:01:04.569431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842829+0000","last_change":"2024-02-14T17:17:19.909867+0000","last_active":"2024-02-14T17:17:43.842829+0000","last_peered":"2024-02-14T17:17:43.842829+0000","last_clean":"2024-02-14T17:17:43.842829+0000","last_became_active":"2024-02-14T17:17:19.909274+0000","last_became_peered":"2024-02-14T17:17:19.909274+0000","last_unstale":"2024-02-14T17:17:43.842829+0000","last_undegraded":"2024-02-14T17:17:43.842829+0000","last_fullsized":"2024-02-14T17:17:43.842829+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:29:50.359669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845315+0000","last_change":"2024-02-14T17:17:15.887941+0000","last_active":"2024-02-14T17:17:43.845315+0000","last_peered":"2024-02-14T17:17:43.845315+0000","last_clean":"2024-02-14T17:17:43.845315+0000","last_became_active":"2024-02-14T17:17:15.886905+0000","last_became_peered":"2024-02-14T17:17:15.886905+0000","last_unstale":"2024-02-14T17:17:43.845315+0000","last_undegraded":"2024-02-14T17:17:43.845315+0000","last_fullsized":"2024-02-14T17:17:43.845315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:39:48.099305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843512+0000","last_change":"2024-02-14T17:17:17.898290+0000","last_active":"2024-02-14T17:17:43.843512+0000","last_peered":"2024-02-14T17:17:43.843512+0000","last_clean":"2024-02-14T17:17:43.843512+0000","last_became_active":"2024-02-14T17:17:17.897984+0000","last_became_peered":"2024-02-14T17:17:17.897984+0000","last_unstale":"2024-02-14T17:17:43.843512+0000","last_undegraded":"2024-02-14T17:17:43.843512+0000","last_fullsized":"2024-02-14T17:17:43.843512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:21:05.388054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'2","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844035+0000","last_change":"2024-02-14T17:17:21.921435+0000","last_active":"2024-02-14T17:17:43.844035+0000","last_peered":"2024-02-14T17:17:43.844035+0000","last_clean":"2024-02-14T17:17:43.844035+0000","last_became_active":"2024-02-14T17:17:21.920977+0000","last_became_peered":"2024-02-14T17:17:21.920977+0000","last_unstale":"2024-02-14T17:17:43.844035+0000","last_undegraded":"2024-02-14T17:17:43.844035+0000","last_fullsized":"2024-02-14T17:17:43.844035+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:47:13.144506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843094+0000","last_change":"2024-02-14T17:17:19.906153+0000","last_active":"2024-02-14T17:17:43.843094+0000","last_peered":"2024-02-14T17:17:43.843094+0000","last_clean":"2024-02-14T17:17:43.843094+0000","last_became_active":"2024-02-14T17:17:19.905424+0000","last_became_peered":"2024-02-14T17:17:19.905424+0000","last_unstale":"2024-02-14T17:17:43.843094+0000","last_undegraded":"2024-02-14T17:17:43.843094+0000","last_fullsized":"2024-02-14T17:17:43.843094+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:27:42.210427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845451+0000","last_change":"2024-02-14T17:17:15.886593+0000","last_active":"2024-02-14T17:17:43.845451+0000","last_peered":"2024-02-14T17:17:43.845451+0000","last_clean":"2024-02-14T17:17:43.845451+0000","last_became_active":"2024-02-14T17:17:15.886184+0000","last_became_peered":"2024-02-14T17:17:15.886184+0000","last_unstale":"2024-02-14T17:17:43.845451+0000","last_undegraded":"2024-02-14T17:17:43.845451+0000","last_fullsized":"2024-02-14T17:17:43.845451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:29:36.526730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845150+0000","last_change":"2024-02-14T17:17:17.897527+0000","last_active":"2024-02-14T17:17:43.845150+0000","last_peered":"2024-02-14T17:17:43.845150+0000","last_clean":"2024-02-14T17:17:43.845150+0000","last_became_active":"2024-02-14T17:17:17.896690+0000","last_became_peered":"2024-02-14T17:17:17.896690+0000","last_unstale":"2024-02-14T17:17:43.845150+0000","last_undegraded":"2024-02-14T17:17:43.845150+0000","last_fullsized":"2024-02-14T17:17:43.845150+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:22:22.689895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844274+0000","last_change":"2024-02-14T17:17:21.926339+0000","last_active":"2024-02-14T17:17:43.844274+0000","last_peered":"2024-02-14T17:17:43.844274+0000","last_clean":"2024-02-14T17:17:43.844274+0000","last_became_active":"2024-02-14T17:17:21.925921+0000","last_became_peered":"2024-02-14T17:17:21.925921+0000","last_unstale":"2024-02-14T17:17:43.844274+0000","last_undegraded":"2024-02-14T17:17:43.844274+0000","last_fullsized":"2024-02-14T17:17:43.844274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:16:56.635263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844406+0000","last_change":"2024-02-14T17:17:19.906688+0000","last_active":"2024-02-14T17:17:43.844406+0000","last_peered":"2024-02-14T17:17:43.844406+0000","last_clean":"2024-02-14T17:17:43.844406+0000","last_became_active":"2024-02-14T17:17:19.906016+0000","last_became_peered":"2024-02-14T17:17:19.906016+0000","last_unstale":"2024-02-14T17:17:43.844406+0000","last_undegraded":"2024-02-14T17:17:43.844406+0000","last_fullsized":"2024-02-14T17:17:43.844406+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:44:51.700864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843225+0000","last_change":"2024-02-14T17:17:15.884881+0000","last_active":"2024-02-14T17:17:43.843225+0000","last_peered":"2024-02-14T17:17:43.843225+0000","last_clean":"2024-02-14T17:17:43.843225+0000","last_became_active":"2024-02-14T17:17:15.884474+0000","last_became_peered":"2024-02-14T17:17:15.884474+0000","last_unstale":"2024-02-14T17:17:43.843225+0000","last_undegraded":"2024-02-14T17:17:43.843225+0000","last_fullsized":"2024-02-14T17:17:43.843225+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:24:40.731728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845127+0000","last_change":"2024-02-14T17:17:17.896663+0000","last_active":"2024-02-14T17:17:43.845127+0000","last_peered":"2024-02-14T17:17:43.845127+0000","last_clean":"2024-02-14T17:17:43.845127+0000","last_became_active":"2024-02-14T17:17:17.896034+0000","last_became_peered":"2024-02-14T17:17:17.896034+0000","last_unstale":"2024-02-14T17:17:43.845127+0000","last_undegraded":"2024-02-14T17:17:43.845127+0000","last_fullsized":"2024-02-14T17:17:43.845127+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:43:40.345511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844020+0000","last_change":"2024-02-14T17:17:21.926167+0000","last_active":"2024-02-14T17:17:43.844020+0000","last_peered":"2024-02-14T17:17:43.844020+0000","last_clean":"2024-02-14T17:17:43.844020+0000","last_became_active":"2024-02-14T17:17:21.925624+0000","last_became_peered":"2024-02-14T17:17:21.925624+0000","last_unstale":"2024-02-14T17:17:43.844020+0000","last_undegraded":"2024-02-14T17:17:43.844020+0000","last_fullsized":"2024-02-14T17:17:43.844020+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:55:25.645101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929716+0000","last_change":"2024-02-14T17:17:19.909069+0000","last_active":"2024-02-14T17:17:45.929716+0000","last_peered":"2024-02-14T17:17:45.929716+0000","last_clean":"2024-02-14T17:17:45.929716+0000","last_became_active":"2024-02-14T17:17:19.908326+0000","last_became_peered":"2024-02-14T17:17:19.908326+0000","last_unstale":"2024-02-14T17:17:45.929716+0000","last_undegraded":"2024-02-14T17:17:45.929716+0000","last_fullsized":"2024-02-14T17:17:45.929716+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:57:01.243136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841845+0000","last_change":"2024-02-14T17:17:15.885539+0000","last_active":"2024-02-14T17:17:43.841845+0000","last_peered":"2024-02-14T17:17:43.841845+0000","last_clean":"2024-02-14T17:17:43.841845+0000","last_became_active":"2024-02-14T17:17:15.885267+0000","last_became_peered":"2024-02-14T17:17:15.885267+0000","last_unstale":"2024-02-14T17:17:43.841845+0000","last_undegraded":"2024-02-14T17:17:43.841845+0000","last_fullsized":"2024-02-14T17:17:43.841845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:45:52.655197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841450+0000","last_change":"2024-02-14T17:17:17.892510+0000","last_active":"2024-02-14T17:17:43.841450+0000","last_peered":"2024-02-14T17:17:43.841450+0000","last_clean":"2024-02-14T17:17:43.841450+0000","last_became_active":"2024-02-14T17:17:17.892143+0000","last_became_peered":"2024-02-14T17:17:17.892143+0000","last_unstale":"2024-02-14T17:17:43.841450+0000","last_undegraded":"2024-02-14T17:17:43.841450+0000","last_fullsized":"2024-02-14T17:17:43.841450+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:50:58.816774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844450+0000","last_change":"2024-02-14T17:17:21.923968+0000","last_active":"2024-02-14T17:17:43.844450+0000","last_peered":"2024-02-14T17:17:43.844450+0000","last_clean":"2024-02-14T17:17:43.844450+0000","last_became_active":"2024-02-14T17:17:21.921806+0000","last_became_peered":"2024-02-14T17:17:21.921806+0000","last_unstale":"2024-02-14T17:17:43.844450+0000","last_undegraded":"2024-02-14T17:17:43.844450+0000","last_fullsized":"2024-02-14T17:17:43.844450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:30:28.966847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844298+0000","last_change":"2024-02-14T17:17:19.910353+0000","last_active":"2024-02-14T17:17:43.844298+0000","last_peered":"2024-02-14T17:17:43.844298+0000","last_clean":"2024-02-14T17:17:43.844298+0000","last_became_active":"2024-02-14T17:17:19.910020+0000","last_became_peered":"2024-02-14T17:17:19.910020+0000","last_unstale":"2024-02-14T17:17:43.844298+0000","last_undegraded":"2024-02-14T17:17:43.844298+0000","last_fullsized":"2024-02-14T17:17:43.844298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:47:15.960147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842445+0000","last_change":"2024-02-14T17:17:15.886401+0000","last_active":"2024-02-14T17:17:43.842445+0000","last_peered":"2024-02-14T17:17:43.842445+0000","last_clean":"2024-02-14T17:17:43.842445+0000","last_became_active":"2024-02-14T17:17:15.885918+0000","last_became_peered":"2024-02-14T17:17:15.885918+0000","last_unstale":"2024-02-14T17:17:43.842445+0000","last_undegraded":"2024-02-14T17:17:43.842445+0000","last_fullsized":"2024-02-14T17:17:43.842445+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:55:40.012434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841501+0000","last_change":"2024-02-14T17:17:17.889690+0000","last_active":"2024-02-14T17:17:43.841501+0000","last_peered":"2024-02-14T17:17:43.841501+0000","last_clean":"2024-02-14T17:17:43.841501+0000","last_became_active":"2024-02-14T17:17:17.889371+0000","last_became_peered":"2024-02-14T17:17:17.889371+0000","last_unstale":"2024-02-14T17:17:43.841501+0000","last_undegraded":"2024-02-14T17:17:43.841501+0000","last_fullsized":"2024-02-14T17:17:43.841501+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:46:42.045637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842501+0000","last_change":"2024-02-14T17:17:21.924013+0000","last_active":"2024-02-14T17:17:43.842501+0000","last_peered":"2024-02-14T17:17:43.842501+0000","last_clean":"2024-02-14T17:17:43.842501+0000","last_became_active":"2024-02-14T17:17:21.922231+0000","last_became_peered":"2024-02-14T17:17:21.922231+0000","last_unstale":"2024-02-14T17:17:43.842501+0000","last_undegraded":"2024-02-14T17:17:43.842501+0000","last_fullsized":"2024-02-14T17:17:43.842501+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:34:11.654557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841427+0000","last_change":"2024-02-14T17:17:19.911419+0000","last_active":"2024-02-14T17:17:43.841427+0000","last_peered":"2024-02-14T17:17:43.841427+0000","last_clean":"2024-02-14T17:17:43.841427+0000","last_became_active":"2024-02-14T17:17:19.910804+0000","last_became_peered":"2024-02-14T17:17:19.910804+0000","last_unstale":"2024-02-14T17:17:43.841427+0000","last_undegraded":"2024-02-14T17:17:43.841427+0000","last_fullsized":"2024-02-14T17:17:43.841427+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:32:34.823529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845372+0000","last_change":"2024-02-14T17:17:15.887700+0000","last_active":"2024-02-14T17:17:43.845372+0000","last_peered":"2024-02-14T17:17:43.845372+0000","last_clean":"2024-02-14T17:17:43.845372+0000","last_became_active":"2024-02-14T17:17:15.886138+0000","last_became_peered":"2024-02-14T17:17:15.886138+0000","last_unstale":"2024-02-14T17:17:43.845372+0000","last_undegraded":"2024-02-14T17:17:43.845372+0000","last_fullsized":"2024-02-14T17:17:43.845372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:55:35.680409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845406+0000","last_change":"2024-02-14T17:17:17.893826+0000","last_active":"2024-02-14T17:17:43.845406+0000","last_peered":"2024-02-14T17:17:43.845406+0000","last_clean":"2024-02-14T17:17:43.845406+0000","last_became_active":"2024-02-14T17:17:17.893412+0000","last_became_peered":"2024-02-14T17:17:17.893412+0000","last_unstale":"2024-02-14T17:17:43.845406+0000","last_undegraded":"2024-02-14T17:17:43.845406+0000","last_fullsized":"2024-02-14T17:17:43.845406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:44:26.910135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842392+0000","last_change":"2024-02-14T17:17:21.921625+0000","last_active":"2024-02-14T17:17:43.842392+0000","last_peered":"2024-02-14T17:17:43.842392+0000","last_clean":"2024-02-14T17:17:43.842392+0000","last_became_active":"2024-02-14T17:17:21.920249+0000","last_became_peered":"2024-02-14T17:17:21.920249+0000","last_unstale":"2024-02-14T17:17:43.842392+0000","last_undegraded":"2024-02-14T17:17:43.842392+0000","last_fullsized":"2024-02-14T17:17:43.842392+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:28:35.624949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841607+0000","last_change":"2024-02-14T17:17:19.907668+0000","last_active":"2024-02-14T17:17:43.841607+0000","last_peered":"2024-02-14T17:17:43.841607+0000","last_clean":"2024-02-14T17:17:43.841607+0000","last_became_active":"2024-02-14T17:17:19.907461+0000","last_became_peered":"2024-02-14T17:17:19.907461+0000","last_unstale":"2024-02-14T17:17:43.841607+0000","last_undegraded":"2024-02-14T17:17:43.841607+0000","last_fullsized":"2024-02-14T17:17:43.841607+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:19:28.580967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845152+0000","last_change":"2024-02-14T17:17:15.886720+0000","last_active":"2024-02-14T17:17:43.845152+0000","last_peered":"2024-02-14T17:17:43.845152+0000","last_clean":"2024-02-14T17:17:43.845152+0000","last_became_active":"2024-02-14T17:17:15.886129+0000","last_became_peered":"2024-02-14T17:17:15.886129+0000","last_unstale":"2024-02-14T17:17:43.845152+0000","last_undegraded":"2024-02-14T17:17:43.845152+0000","last_fullsized":"2024-02-14T17:17:43.845152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:05:56.680074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846235+0000","last_change":"2024-02-14T17:17:17.897144+0000","last_active":"2024-02-14T17:17:43.846235+0000","last_peered":"2024-02-14T17:17:43.846235+0000","last_clean":"2024-02-14T17:17:43.846235+0000","last_became_active":"2024-02-14T17:17:17.896665+0000","last_became_peered":"2024-02-14T17:17:17.896665+0000","last_unstale":"2024-02-14T17:17:43.846235+0000","last_undegraded":"2024-02-14T17:17:43.846235+0000","last_fullsized":"2024-02-14T17:17:43.846235+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:49:25.852754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843686+0000","last_change":"2024-02-14T17:17:21.920499+0000","last_active":"2024-02-14T17:17:43.843686+0000","last_peered":"2024-02-14T17:17:43.843686+0000","last_clean":"2024-02-14T17:17:43.843686+0000","last_became_active":"2024-02-14T17:17:21.920204+0000","last_became_peered":"2024-02-14T17:17:21.920204+0000","last_unstale":"2024-02-14T17:17:43.843686+0000","last_undegraded":"2024-02-14T17:17:43.843686+0000","last_fullsized":"2024-02-14T17:17:43.843686+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:02:29.697540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842893+0000","last_change":"2024-02-14T17:17:19.909249+0000","last_active":"2024-02-14T17:17:43.842893+0000","last_peered":"2024-02-14T17:17:43.842893+0000","last_clean":"2024-02-14T17:17:43.842893+0000","last_became_active":"2024-02-14T17:17:19.908670+0000","last_became_peered":"2024-02-14T17:17:19.908670+0000","last_unstale":"2024-02-14T17:17:43.842893+0000","last_undegraded":"2024-02-14T17:17:43.842893+0000","last_fullsized":"2024-02-14T17:17:43.842893+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:09:16.040128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845394+0000","last_change":"2024-02-14T17:17:15.888261+0000","last_active":"2024-02-14T17:17:43.845394+0000","last_peered":"2024-02-14T17:17:43.845394+0000","last_clean":"2024-02-14T17:17:43.845394+0000","last_became_active":"2024-02-14T17:17:15.887978+0000","last_became_peered":"2024-02-14T17:17:15.887978+0000","last_unstale":"2024-02-14T17:17:43.845394+0000","last_undegraded":"2024-02-14T17:17:43.845394+0000","last_fullsized":"2024-02-14T17:17:43.845394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:13.031412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844889+0000","last_change":"2024-02-14T17:17:17.892929+0000","last_active":"2024-02-14T17:17:43.844889+0000","last_peered":"2024-02-14T17:17:43.844889+0000","last_clean":"2024-02-14T17:17:43.844889+0000","last_became_active":"2024-02-14T17:17:17.891909+0000","last_became_peered":"2024-02-14T17:17:17.891909+0000","last_unstale":"2024-02-14T17:17:43.844889+0000","last_undegraded":"2024-02-14T17:17:43.844889+0000","last_fullsized":"2024-02-14T17:17:43.844889+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:38:00.734469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842863+0000","last_change":"2024-02-14T17:17:21.920505+0000","last_active":"2024-02-14T17:17:43.842863+0000","last_peered":"2024-02-14T17:17:43.842863+0000","last_clean":"2024-02-14T17:17:43.842863+0000","last_became_active":"2024-02-14T17:17:21.920171+0000","last_became_peered":"2024-02-14T17:17:21.920171+0000","last_unstale":"2024-02-14T17:17:43.842863+0000","last_undegraded":"2024-02-14T17:17:43.842863+0000","last_fullsized":"2024-02-14T17:17:43.842863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:44:26.244773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842180+0000","last_change":"2024-02-14T17:17:19.911072+0000","last_active":"2024-02-14T17:17:43.842180+0000","last_peered":"2024-02-14T17:17:43.842180+0000","last_clean":"2024-02-14T17:17:43.842180+0000","last_became_active":"2024-02-14T17:17:19.910665+0000","last_became_peered":"2024-02-14T17:17:19.910665+0000","last_unstale":"2024-02-14T17:17:43.842180+0000","last_undegraded":"2024-02-14T17:17:43.842180+0000","last_fullsized":"2024-02-14T17:17:43.842180+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:04:28.401412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844172+0000","last_change":"2024-02-14T17:17:15.886827+0000","last_active":"2024-02-14T17:17:43.844172+0000","last_peered":"2024-02-14T17:17:43.844172+0000","last_clean":"2024-02-14T17:17:43.844172+0000","last_became_active":"2024-02-14T17:17:15.886068+0000","last_became_peered":"2024-02-14T17:17:15.886068+0000","last_unstale":"2024-02-14T17:17:43.844172+0000","last_undegraded":"2024-02-14T17:17:43.844172+0000","last_fullsized":"2024-02-14T17:17:43.844172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:27:32.304159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843655+0000","last_change":"2024-02-14T17:17:17.897817+0000","last_active":"2024-02-14T17:17:43.843655+0000","last_peered":"2024-02-14T17:17:43.843655+0000","last_clean":"2024-02-14T17:17:43.843655+0000","last_became_active":"2024-02-14T17:17:17.897317+0000","last_became_peered":"2024-02-14T17:17:17.897317+0000","last_unstale":"2024-02-14T17:17:43.843655+0000","last_undegraded":"2024-02-14T17:17:43.843655+0000","last_fullsized":"2024-02-14T17:17:43.843655+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:45:52.526374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844914+0000","last_change":"2024-02-14T17:17:21.925924+0000","last_active":"2024-02-14T17:17:43.844914+0000","last_peered":"2024-02-14T17:17:43.844914+0000","last_clean":"2024-02-14T17:17:43.844914+0000","last_became_active":"2024-02-14T17:17:21.925513+0000","last_became_peered":"2024-02-14T17:17:21.925513+0000","last_unstale":"2024-02-14T17:17:43.844914+0000","last_undegraded":"2024-02-14T17:17:43.844914+0000","last_fullsized":"2024-02-14T17:17:43.844914+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:36:29.218922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929710+0000","last_change":"2024-02-14T17:17:19.908714+0000","last_active":"2024-02-14T17:17:45.929710+0000","last_peered":"2024-02-14T17:17:45.929710+0000","last_clean":"2024-02-14T17:17:45.929710+0000","last_became_active":"2024-02-14T17:17:19.908482+0000","last_became_peered":"2024-02-14T17:17:19.908482+0000","last_unstale":"2024-02-14T17:17:45.929710+0000","last_undegraded":"2024-02-14T17:17:45.929710+0000","last_fullsized":"2024-02-14T17:17:45.929710+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:35:22.151972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845530+0000","last_change":"2024-02-14T17:17:15.887365+0000","last_active":"2024-02-14T17:17:43.845530+0000","last_peered":"2024-02-14T17:17:43.845530+0000","last_clean":"2024-02-14T17:17:43.845530+0000","last_became_active":"2024-02-14T17:17:15.886493+0000","last_became_peered":"2024-02-14T17:17:15.886493+0000","last_unstale":"2024-02-14T17:17:43.845530+0000","last_undegraded":"2024-02-14T17:17:43.845530+0000","last_fullsized":"2024-02-14T17:17:43.845530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:42:40.980118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842433+0000","last_change":"2024-02-14T17:17:17.889757+0000","last_active":"2024-02-14T17:17:43.842433+0000","last_peered":"2024-02-14T17:17:43.842433+0000","last_clean":"2024-02-14T17:17:43.842433+0000","last_became_active":"2024-02-14T17:17:17.889413+0000","last_became_peered":"2024-02-14T17:17:17.889413+0000","last_unstale":"2024-02-14T17:17:43.842433+0000","last_undegraded":"2024-02-14T17:17:43.842433+0000","last_fullsized":"2024-02-14T17:17:43.842433+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:18:26.829865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844012+0000","last_change":"2024-02-14T17:17:21.922052+0000","last_active":"2024-02-14T17:17:43.844012+0000","last_peered":"2024-02-14T17:17:43.844012+0000","last_clean":"2024-02-14T17:17:43.844012+0000","last_became_active":"2024-02-14T17:17:21.920969+0000","last_became_peered":"2024-02-14T17:17:21.920969+0000","last_unstale":"2024-02-14T17:17:43.844012+0000","last_undegraded":"2024-02-14T17:17:43.844012+0000","last_fullsized":"2024-02-14T17:17:43.844012+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:04:17.155323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929684+0000","last_change":"2024-02-14T17:17:19.908580+0000","last_active":"2024-02-14T17:17:45.929684+0000","last_peered":"2024-02-14T17:17:45.929684+0000","last_clean":"2024-02-14T17:17:45.929684+0000","last_became_active":"2024-02-14T17:17:19.908385+0000","last_became_peered":"2024-02-14T17:17:19.908385+0000","last_unstale":"2024-02-14T17:17:45.929684+0000","last_undegraded":"2024-02-14T17:17:45.929684+0000","last_fullsized":"2024-02-14T17:17:45.929684+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:39:04.978103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843940+0000","last_change":"2024-02-14T17:17:15.886323+0000","last_active":"2024-02-14T17:17:43.843940+0000","last_peered":"2024-02-14T17:17:43.843940+0000","last_clean":"2024-02-14T17:17:43.843940+0000","last_became_active":"2024-02-14T17:17:15.885955+0000","last_became_peered":"2024-02-14T17:17:15.885955+0000","last_unstale":"2024-02-14T17:17:43.843940+0000","last_undegraded":"2024-02-14T17:17:43.843940+0000","last_fullsized":"2024-02-14T17:17:43.843940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:13:22.299558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843121+0000","last_change":"2024-02-14T17:17:17.894466+0000","last_active":"2024-02-14T17:17:43.843121+0000","last_peered":"2024-02-14T17:17:43.843121+0000","last_clean":"2024-02-14T17:17:43.843121+0000","last_became_active":"2024-02-14T17:17:17.893598+0000","last_became_peered":"2024-02-14T17:17:17.893598+0000","last_unstale":"2024-02-14T17:17:43.843121+0000","last_undegraded":"2024-02-14T17:17:43.843121+0000","last_fullsized":"2024-02-14T17:17:43.843121+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:17:56.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844357+0000","last_change":"2024-02-14T17:17:21.921966+0000","last_active":"2024-02-14T17:17:43.844357+0000","last_peered":"2024-02-14T17:17:43.844357+0000","last_clean":"2024-02-14T17:17:43.844357+0000","last_became_active":"2024-02-14T17:17:21.921232+0000","last_became_peered":"2024-02-14T17:17:21.921232+0000","last_unstale":"2024-02-14T17:17:43.844357+0000","last_undegraded":"2024-02-14T17:17:43.844357+0000","last_fullsized":"2024-02-14T17:17:43.844357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:21:18.438708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845754+0000","last_change":"2024-02-14T17:17:19.910485+0000","last_active":"2024-02-14T17:17:43.845754+0000","last_peered":"2024-02-14T17:17:43.845754+0000","last_clean":"2024-02-14T17:17:43.845754+0000","last_became_active":"2024-02-14T17:17:19.910143+0000","last_became_peered":"2024-02-14T17:17:19.910143+0000","last_unstale":"2024-02-14T17:17:43.845754+0000","last_undegraded":"2024-02-14T17:17:43.845754+0000","last_fullsized":"2024-02-14T17:17:43.845754+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:43:50.628165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842293+0000","last_change":"2024-02-14T17:17:15.886456+0000","last_active":"2024-02-14T17:17:43.842293+0000","last_peered":"2024-02-14T17:17:43.842293+0000","last_clean":"2024-02-14T17:17:43.842293+0000","last_became_active":"2024-02-14T17:17:15.886006+0000","last_became_peered":"2024-02-14T17:17:15.886006+0000","last_unstale":"2024-02-14T17:17:43.842293+0000","last_undegraded":"2024-02-14T17:17:43.842293+0000","last_fullsized":"2024-02-14T17:17:43.842293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:01:37.200369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841888+0000","last_change":"2024-02-14T17:17:17.895636+0000","last_active":"2024-02-14T17:17:43.841888+0000","last_peered":"2024-02-14T17:17:43.841888+0000","last_clean":"2024-02-14T17:17:43.841888+0000","last_became_active":"2024-02-14T17:17:17.895345+0000","last_became_peered":"2024-02-14T17:17:17.895345+0000","last_unstale":"2024-02-14T17:17:43.841888+0000","last_undegraded":"2024-02-14T17:17:43.841888+0000","last_fullsized":"2024-02-14T17:17:43.841888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:14:36.840742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845496+0000","last_change":"2024-02-14T17:17:21.926444+0000","last_active":"2024-02-14T17:17:43.845496+0000","last_peered":"2024-02-14T17:17:43.845496+0000","last_clean":"2024-02-14T17:17:43.845496+0000","last_became_active":"2024-02-14T17:17:21.926054+0000","last_became_peered":"2024-02-14T17:17:21.926054+0000","last_unstale":"2024-02-14T17:17:43.845496+0000","last_undegraded":"2024-02-14T17:17:43.845496+0000","last_fullsized":"2024-02-14T17:17:43.845496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:48:44.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844386+0000","last_change":"2024-02-14T17:17:19.911233+0000","last_active":"2024-02-14T17:17:43.844386+0000","last_peered":"2024-02-14T17:17:43.844386+0000","last_clean":"2024-02-14T17:17:43.844386+0000","last_became_active":"2024-02-14T17:17:19.910829+0000","last_became_peered":"2024-02-14T17:17:19.910829+0000","last_unstale":"2024-02-14T17:17:43.844386+0000","last_undegraded":"2024-02-14T17:17:43.844386+0000","last_fullsized":"2024-02-14T17:17:43.844386+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:48:25.914208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842659+0000","last_change":"2024-02-14T17:17:15.888379+0000","last_active":"2024-02-14T17:17:43.842659+0000","last_peered":"2024-02-14T17:17:43.842659+0000","last_clean":"2024-02-14T17:17:43.842659+0000","last_became_active":"2024-02-14T17:17:15.886610+0000","last_became_peered":"2024-02-14T17:17:15.886610+0000","last_unstale":"2024-02-14T17:17:43.842659+0000","last_undegraded":"2024-02-14T17:17:43.842659+0000","last_fullsized":"2024-02-14T17:17:43.842659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:18:50.359270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842770+0000","last_change":"2024-02-14T17:17:17.897174+0000","last_active":"2024-02-14T17:17:43.842770+0000","last_peered":"2024-02-14T17:17:43.842770+0000","last_clean":"2024-02-14T17:17:43.842770+0000","last_became_active":"2024-02-14T17:17:17.896118+0000","last_became_peered":"2024-02-14T17:17:17.896118+0000","last_unstale":"2024-02-14T17:17:43.842770+0000","last_undegraded":"2024-02-14T17:17:43.842770+0000","last_fullsized":"2024-02-14T17:17:43.842770+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:56:40.460111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845151+0000","last_change":"2024-02-14T17:17:21.926390+0000","last_active":"2024-02-14T17:17:43.845151+0000","last_peered":"2024-02-14T17:17:43.845151+0000","last_clean":"2024-02-14T17:17:43.845151+0000","last_became_active":"2024-02-14T17:17:21.925965+0000","last_became_peered":"2024-02-14T17:17:21.925965+0000","last_unstale":"2024-02-14T17:17:43.845151+0000","last_undegraded":"2024-02-14T17:17:43.845151+0000","last_fullsized":"2024-02-14T17:17:43.845151+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:11:58.289687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844298+0000","last_change":"2024-02-14T17:17:15.886283+0000","last_active":"2024-02-14T17:17:43.844298+0000","last_peered":"2024-02-14T17:17:43.844298+0000","last_clean":"2024-02-14T17:17:43.844298+0000","last_became_active":"2024-02-14T17:17:15.885808+0000","last_became_peered":"2024-02-14T17:17:15.885808+0000","last_unstale":"2024-02-14T17:17:43.844298+0000","last_undegraded":"2024-02-14T17:17:43.844298+0000","last_fullsized":"2024-02-14T17:17:43.844298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:45:14.411993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844264+0000","last_change":"2024-02-14T17:17:17.898169+0000","last_active":"2024-02-14T17:17:43.844264+0000","last_peered":"2024-02-14T17:17:43.844264+0000","last_clean":"2024-02-14T17:17:43.844264+0000","last_became_active":"2024-02-14T17:17:17.897377+0000","last_became_peered":"2024-02-14T17:17:17.897377+0000","last_unstale":"2024-02-14T17:17:43.844264+0000","last_undegraded":"2024-02-14T17:17:43.844264+0000","last_fullsized":"2024-02-14T17:17:43.844264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:23:58.178604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929739+0000","last_change":"2024-02-14T17:17:19.906734+0000","last_active":"2024-02-14T17:17:45.929739+0000","last_peered":"2024-02-14T17:17:45.929739+0000","last_clean":"2024-02-14T17:17:45.929739+0000","last_became_active":"2024-02-14T17:17:19.906024+0000","last_became_peered":"2024-02-14T17:17:19.906024+0000","last_unstale":"2024-02-14T17:17:45.929739+0000","last_undegraded":"2024-02-14T17:17:45.929739+0000","last_fullsized":"2024-02-14T17:17:45.929739+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:28:55.748001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843220+0000","last_change":"2024-02-14T17:17:21.921812+0000","last_active":"2024-02-14T17:17:43.843220+0000","last_peered":"2024-02-14T17:17:43.843220+0000","last_clean":"2024-02-14T17:17:43.843220+0000","last_became_active":"2024-02-14T17:17:21.921500+0000","last_became_peered":"2024-02-14T17:17:21.921500+0000","last_unstale":"2024-02-14T17:17:43.843220+0000","last_undegraded":"2024-02-14T17:17:43.843220+0000","last_fullsized":"2024-02-14T17:17:43.843220+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:48.154003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845252+0000","last_change":"2024-02-14T17:17:15.885041+0000","last_active":"2024-02-14T17:17:43.845252+0000","last_peered":"2024-02-14T17:17:43.845252+0000","last_clean":"2024-02-14T17:17:43.845252+0000","last_became_active":"2024-02-14T17:17:15.884070+0000","last_became_peered":"2024-02-14T17:17:15.884070+0000","last_unstale":"2024-02-14T17:17:43.845252+0000","last_undegraded":"2024-02-14T17:17:43.845252+0000","last_fullsized":"2024-02-14T17:17:43.845252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:17:02.064818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:50.930949+0000","last_change":"2024-02-14T17:17:17.892591+0000","last_active":"2024-02-14T17:17:50.930949+0000","last_peered":"2024-02-14T17:17:50.930949+0000","last_clean":"2024-02-14T17:17:50.930949+0000","last_became_active":"2024-02-14T17:17:17.892127+0000","last_became_peered":"2024-02-14T17:17:17.892127+0000","last_unstale":"2024-02-14T17:17:50.930949+0000","last_undegraded":"2024-02-14T17:17:50.930949+0000","last_fullsized":"2024-02-14T17:17:50.930949+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:09:52.160369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843779+0000","last_change":"2024-02-14T17:17:19.906012+0000","last_active":"2024-02-14T17:17:43.843779+0000","last_peered":"2024-02-14T17:17:43.843779+0000","last_clean":"2024-02-14T17:17:43.843779+0000","last_became_active":"2024-02-14T17:17:19.905585+0000","last_became_peered":"2024-02-14T17:17:19.905585+0000","last_unstale":"2024-02-14T17:17:43.843779+0000","last_undegraded":"2024-02-14T17:17:43.843779+0000","last_fullsized":"2024-02-14T17:17:43.843779+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:46:39.095722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843121+0000","last_change":"2024-02-14T17:17:21.925826+0000","last_active":"2024-02-14T17:17:43.843121+0000","last_peered":"2024-02-14T17:17:43.843121+0000","last_clean":"2024-02-14T17:17:43.843121+0000","last_became_active":"2024-02-14T17:17:21.925438+0000","last_became_peered":"2024-02-14T17:17:21.925438+0000","last_unstale":"2024-02-14T17:17:43.843121+0000","last_undegraded":"2024-02-14T17:17:43.843121+0000","last_fullsized":"2024-02-14T17:17:43.843121+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:14:15.811566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843371+0000","last_change":"2024-02-14T17:17:15.884735+0000","last_active":"2024-02-14T17:17:43.843371+0000","last_peered":"2024-02-14T17:17:43.843371+0000","last_clean":"2024-02-14T17:17:43.843371+0000","last_became_active":"2024-02-14T17:17:15.884347+0000","last_became_peered":"2024-02-14T17:17:15.884347+0000","last_unstale":"2024-02-14T17:17:43.843371+0000","last_undegraded":"2024-02-14T17:17:43.843371+0000","last_fullsized":"2024-02-14T17:17:43.843371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:37:57.983734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843198+0000","last_change":"2024-02-14T17:17:17.893817+0000","last_active":"2024-02-14T17:17:43.843198+0000","last_peered":"2024-02-14T17:17:43.843198+0000","last_clean":"2024-02-14T17:17:43.843198+0000","last_became_active":"2024-02-14T17:17:17.892935+0000","last_became_peered":"2024-02-14T17:17:17.892935+0000","last_unstale":"2024-02-14T17:17:43.843198+0000","last_undegraded":"2024-02-14T17:17:43.843198+0000","last_fullsized":"2024-02-14T17:17:43.843198+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:07:50.191051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844939+0000","last_change":"2024-02-14T17:17:19.912172+0000","last_active":"2024-02-14T17:17:43.844939+0000","last_peered":"2024-02-14T17:17:43.844939+0000","last_clean":"2024-02-14T17:17:43.844939+0000","last_became_active":"2024-02-14T17:17:19.911528+0000","last_became_peered":"2024-02-14T17:17:19.911528+0000","last_unstale":"2024-02-14T17:17:43.844939+0000","last_undegraded":"2024-02-14T17:17:43.844939+0000","last_fullsized":"2024-02-14T17:17:43.844939+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:14:25.458332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845191+0000","last_change":"2024-02-14T17:17:21.926039+0000","last_active":"2024-02-14T17:17:43.845191+0000","last_peered":"2024-02-14T17:17:43.845191+0000","last_clean":"2024-02-14T17:17:43.845191+0000","last_became_active":"2024-02-14T17:17:21.925727+0000","last_became_peered":"2024-02-14T17:17:21.925727+0000","last_unstale":"2024-02-14T17:17:43.845191+0000","last_undegraded":"2024-02-14T17:17:43.845191+0000","last_fullsized":"2024-02-14T17:17:43.845191+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:34:38.358980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842771+0000","last_change":"2024-02-14T17:17:17.898095+0000","last_active":"2024-02-14T17:17:43.842771+0000","last_peered":"2024-02-14T17:17:43.842771+0000","last_clean":"2024-02-14T17:17:43.842771+0000","last_became_active":"2024-02-14T17:17:17.897231+0000","last_became_peered":"2024-02-14T17:17:17.897231+0000","last_unstale":"2024-02-14T17:17:43.842771+0000","last_undegraded":"2024-02-14T17:17:43.842771+0000","last_fullsized":"2024-02-14T17:17:43.842771+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:55:37.784640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841595+0000","last_change":"2024-02-14T17:17:15.884932+0000","last_active":"2024-02-14T17:17:43.841595+0000","last_peered":"2024-02-14T17:17:43.841595+0000","last_clean":"2024-02-14T17:17:43.841595+0000","last_became_active":"2024-02-14T17:17:15.884205+0000","last_became_peered":"2024-02-14T17:17:15.884205+0000","last_unstale":"2024-02-14T17:17:43.841595+0000","last_undegraded":"2024-02-14T17:17:43.841595+0000","last_fullsized":"2024-02-14T17:17:43.841595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:37:56.312318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846150+0000","last_change":"2024-02-14T17:17:19.912025+0000","last_active":"2024-02-14T17:17:43.846150+0000","last_peered":"2024-02-14T17:17:43.846150+0000","last_clean":"2024-02-14T17:17:43.846150+0000","last_became_active":"2024-02-14T17:17:19.911531+0000","last_became_peered":"2024-02-14T17:17:19.911531+0000","last_unstale":"2024-02-14T17:17:43.846150+0000","last_undegraded":"2024-02-14T17:17:43.846150+0000","last_fullsized":"2024-02-14T17:17:43.846150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:02:25.244444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843065+0000","last_change":"2024-02-14T17:17:21.921276+0000","last_active":"2024-02-14T17:17:43.843065+0000","last_peered":"2024-02-14T17:17:43.843065+0000","last_clean":"2024-02-14T17:17:43.843065+0000","last_became_active":"2024-02-14T17:17:21.920856+0000","last_became_peered":"2024-02-14T17:17:21.920856+0000","last_unstale":"2024-02-14T17:17:43.843065+0000","last_undegraded":"2024-02-14T17:17:43.843065+0000","last_fullsized":"2024-02-14T17:17:43.843065+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:04:49.768478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842758+0000","last_change":"2024-02-14T17:17:17.895183+0000","last_active":"2024-02-14T17:17:43.842758+0000","last_peered":"2024-02-14T17:17:43.842758+0000","last_clean":"2024-02-14T17:17:43.842758+0000","last_became_active":"2024-02-14T17:17:17.894667+0000","last_became_peered":"2024-02-14T17:17:17.894667+0000","last_unstale":"2024-02-14T17:17:43.842758+0000","last_undegraded":"2024-02-14T17:17:43.842758+0000","last_fullsized":"2024-02-14T17:17:43.842758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:10:49.864207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842981+0000","last_change":"2024-02-14T17:17:15.884967+0000","last_active":"2024-02-14T17:17:43.842981+0000","last_peered":"2024-02-14T17:17:43.842981+0000","last_clean":"2024-02-14T17:17:43.842981+0000","last_became_active":"2024-02-14T17:17:15.883693+0000","last_became_peered":"2024-02-14T17:17:15.883693+0000","last_unstale":"2024-02-14T17:17:43.842981+0000","last_undegraded":"2024-02-14T17:17:43.842981+0000","last_fullsized":"2024-02-14T17:17:43.842981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:56:34.889112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.939941+0000","last_change":"2024-02-14T17:17:01.905589+0000","last_active":"2024-02-14T17:17:45.939941+0000","last_peered":"2024-02-14T17:17:45.939941+0000","last_clean":"2024-02-14T17:17:45.939941+0000","last_became_active":"2024-02-14T17:17:01.905307+0000","last_became_peered":"2024-02-14T17:17:01.905307+0000","last_unstale":"2024-02-14T17:17:45.939941+0000","last_undegraded":"2024-02-14T17:17:45.939941+0000","last_fullsized":"2024-02-14T17:17:45.939941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:00.892872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:00.892872+0000","last_clean_scrub_stamp":"2024-02-14T17:17:00.892872+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:14:45.527406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844466+0000","last_change":"2024-02-14T17:17:19.909415+0000","last_active":"2024-02-14T17:17:43.844466+0000","last_peered":"2024-02-14T17:17:43.844466+0000","last_clean":"2024-02-14T17:17:43.844466+0000","last_became_active":"2024-02-14T17:17:19.908694+0000","last_became_peered":"2024-02-14T17:17:19.908694+0000","last_unstale":"2024-02-14T17:17:43.844466+0000","last_undegraded":"2024-02-14T17:17:43.844466+0000","last_fullsized":"2024-02-14T17:17:43.844466+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:25:11.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844537+0000","last_change":"2024-02-14T17:17:21.924435+0000","last_active":"2024-02-14T17:17:43.844537+0000","last_peered":"2024-02-14T17:17:43.844537+0000","last_clean":"2024-02-14T17:17:43.844537+0000","last_became_active":"2024-02-14T17:17:21.922537+0000","last_became_peered":"2024-02-14T17:17:21.922537+0000","last_unstale":"2024-02-14T17:17:43.844537+0000","last_undegraded":"2024-02-14T17:17:43.844537+0000","last_fullsized":"2024-02-14T17:17:43.844537+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:08:46.968206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841920+0000","last_change":"2024-02-14T17:17:17.894386+0000","last_active":"2024-02-14T17:17:43.841920+0000","last_peered":"2024-02-14T17:17:43.841920+0000","last_clean":"2024-02-14T17:17:43.841920+0000","last_became_active":"2024-02-14T17:17:17.894022+0000","last_became_peered":"2024-02-14T17:17:17.894022+0000","last_unstale":"2024-02-14T17:17:43.841920+0000","last_undegraded":"2024-02-14T17:17:43.841920+0000","last_fullsized":"2024-02-14T17:17:43.841920+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:46:42.476920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841892+0000","last_change":"2024-02-14T17:17:15.887911+0000","last_active":"2024-02-14T17:17:43.841892+0000","last_peered":"2024-02-14T17:17:43.841892+0000","last_clean":"2024-02-14T17:17:43.841892+0000","last_became_active":"2024-02-14T17:17:15.887051+0000","last_became_peered":"2024-02-14T17:17:15.887051+0000","last_unstale":"2024-02-14T17:17:43.841892+0000","last_undegraded":"2024-02-14T17:17:43.841892+0000","last_fullsized":"2024-02-14T17:17:43.841892+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:21:32.811193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843507+0000","last_change":"2024-02-14T17:17:19.906830+0000","last_active":"2024-02-14T17:17:43.843507+0000","last_peered":"2024-02-14T17:17:43.843507+0000","last_clean":"2024-02-14T17:17:43.843507+0000","last_became_active":"2024-02-14T17:17:19.906113+0000","last_became_peered":"2024-02-14T17:17:19.906113+0000","last_unstale":"2024-02-14T17:17:43.843507+0000","last_undegraded":"2024-02-14T17:17:43.843507+0000","last_fullsized":"2024-02-14T17:17:43.843507+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:28:01.451955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843127+0000","last_change":"2024-02-14T17:17:21.920517+0000","last_active":"2024-02-14T17:17:43.843127+0000","last_peered":"2024-02-14T17:17:43.843127+0000","last_clean":"2024-02-14T17:17:43.843127+0000","last_became_active":"2024-02-14T17:17:21.920126+0000","last_became_peered":"2024-02-14T17:17:21.920126+0000","last_unstale":"2024-02-14T17:17:43.843127+0000","last_undegraded":"2024-02-14T17:17:43.843127+0000","last_fullsized":"2024-02-14T17:17:43.843127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:11:16.287000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845285+0000","last_change":"2024-02-14T17:17:15.889334+0000","last_active":"2024-02-14T17:17:43.845285+0000","last_peered":"2024-02-14T17:17:43.845285+0000","last_clean":"2024-02-14T17:17:43.845285+0000","last_became_active":"2024-02-14T17:17:15.888605+0000","last_became_peered":"2024-02-14T17:17:15.888605+0000","last_unstale":"2024-02-14T17:17:43.845285+0000","last_undegraded":"2024-02-14T17:17:43.845285+0000","last_fullsized":"2024-02-14T17:17:43.845285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:45:08.389257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":80,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:50.930694+0000","last_change":"2024-02-14T17:17:17.894539+0000","last_active":"2024-02-14T17:17:50.930694+0000","last_peered":"2024-02-14T17:17:50.930694+0000","last_clean":"2024-02-14T17:17:50.930694+0000","last_became_active":"2024-02-14T17:17:17.894127+0000","last_became_peered":"2024-02-14T17:17:17.894127+0000","last_unstale":"2024-02-14T17:17:50.930694+0000","last_undegraded":"2024-02-14T17:17:50.930694+0000","last_fullsized":"2024-02-14T17:17:50.930694+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:18:06.678417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:45.929730+0000","last_change":"2024-02-14T17:17:19.906361+0000","last_active":"2024-02-14T17:17:45.929730+0000","last_peered":"2024-02-14T17:17:45.929730+0000","last_clean":"2024-02-14T17:17:45.929730+0000","last_became_active":"2024-02-14T17:17:19.906018+0000","last_became_peered":"2024-02-14T17:17:19.906018+0000","last_unstale":"2024-02-14T17:17:45.929730+0000","last_undegraded":"2024-02-14T17:17:45.929730+0000","last_fullsized":"2024-02-14T17:17:45.929730+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:24:52.761717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841933+0000","last_change":"2024-02-14T17:17:21.921770+0000","last_active":"2024-02-14T17:17:43.841933+0000","last_peered":"2024-02-14T17:17:43.841933+0000","last_clean":"2024-02-14T17:17:43.841933+0000","last_became_active":"2024-02-14T17:17:21.921354+0000","last_became_peered":"2024-02-14T17:17:21.921354+0000","last_unstale":"2024-02-14T17:17:43.841933+0000","last_undegraded":"2024-02-14T17:17:43.841933+0000","last_fullsized":"2024-02-14T17:17:43.841933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:01:22.458852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845089+0000","last_change":"2024-02-14T17:17:15.885290+0000","last_active":"2024-02-14T17:17:43.845089+0000","last_peered":"2024-02-14T17:17:43.845089+0000","last_clean":"2024-02-14T17:17:43.845089+0000","last_became_active":"2024-02-14T17:17:15.885055+0000","last_became_peered":"2024-02-14T17:17:15.885055+0000","last_unstale":"2024-02-14T17:17:43.845089+0000","last_undegraded":"2024-02-14T17:17:43.845089+0000","last_fullsized":"2024-02-14T17:17:43.845089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:10:09.199166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.846659+0000","last_change":"2024-02-14T17:17:17.897066+0000","last_active":"2024-02-14T17:17:43.846659+0000","last_peered":"2024-02-14T17:17:43.846659+0000","last_clean":"2024-02-14T17:17:43.846659+0000","last_became_active":"2024-02-14T17:17:17.896633+0000","last_became_peered":"2024-02-14T17:17:17.896633+0000","last_unstale":"2024-02-14T17:17:43.846659+0000","last_undegraded":"2024-02-14T17:17:43.846659+0000","last_fullsized":"2024-02-14T17:17:43.846659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:42:12.404731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:50.930693+0000","last_change":"2024-02-14T17:17:19.911895+0000","last_active":"2024-02-14T17:17:50.930693+0000","last_peered":"2024-02-14T17:17:50.930693+0000","last_clean":"2024-02-14T17:17:50.930693+0000","last_became_active":"2024-02-14T17:17:19.911225+0000","last_became_peered":"2024-02-14T17:17:19.911225+0000","last_unstale":"2024-02-14T17:17:50.930693+0000","last_undegraded":"2024-02-14T17:17:50.930693+0000","last_fullsized":"2024-02-14T17:17:50.930693+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:00.612864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844197+0000","last_change":"2024-02-14T17:17:21.926560+0000","last_active":"2024-02-14T17:17:43.844197+0000","last_peered":"2024-02-14T17:17:43.844197+0000","last_clean":"2024-02-14T17:17:43.844197+0000","last_became_active":"2024-02-14T17:17:21.926284+0000","last_became_peered":"2024-02-14T17:17:21.926284+0000","last_unstale":"2024-02-14T17:17:43.844197+0000","last_undegraded":"2024-02-14T17:17:43.844197+0000","last_fullsized":"2024-02-14T17:17:43.844197+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:46.148924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845128+0000","last_change":"2024-02-14T17:17:15.886443+0000","last_active":"2024-02-14T17:17:43.845128+0000","last_peered":"2024-02-14T17:17:43.845128+0000","last_clean":"2024-02-14T17:17:43.845128+0000","last_became_active":"2024-02-14T17:17:15.886060+0000","last_became_peered":"2024-02-14T17:17:15.886060+0000","last_unstale":"2024-02-14T17:17:43.845128+0000","last_undegraded":"2024-02-14T17:17:43.845128+0000","last_fullsized":"2024-02-14T17:17:43.845128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:22:30.648925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844531+0000","last_change":"2024-02-14T17:17:17.894744+0000","last_active":"2024-02-14T17:17:43.844531+0000","last_peered":"2024-02-14T17:17:43.844531+0000","last_clean":"2024-02-14T17:17:43.844531+0000","last_became_active":"2024-02-14T17:17:17.894442+0000","last_became_peered":"2024-02-14T17:17:17.894442+0000","last_unstale":"2024-02-14T17:17:43.844531+0000","last_undegraded":"2024-02-14T17:17:43.844531+0000","last_fullsized":"2024-02-14T17:17:43.844531+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:02.060037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842586+0000","last_change":"2024-02-14T17:17:19.909156+0000","last_active":"2024-02-14T17:17:43.842586+0000","last_peered":"2024-02-14T17:17:43.842586+0000","last_clean":"2024-02-14T17:17:43.842586+0000","last_became_active":"2024-02-14T17:17:19.908967+0000","last_became_peered":"2024-02-14T17:17:19.908967+0000","last_unstale":"2024-02-14T17:17:43.842586+0000","last_undegraded":"2024-02-14T17:17:43.842586+0000","last_fullsized":"2024-02-14T17:17:43.842586+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:50:36.076268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844937+0000","last_change":"2024-02-14T17:17:21.924320+0000","last_active":"2024-02-14T17:17:43.844937+0000","last_peered":"2024-02-14T17:17:43.844937+0000","last_clean":"2024-02-14T17:17:43.844937+0000","last_became_active":"2024-02-14T17:17:21.922224+0000","last_became_peered":"2024-02-14T17:17:21.922224+0000","last_unstale":"2024-02-14T17:17:43.844937+0000","last_undegraded":"2024-02-14T17:17:43.844937+0000","last_fullsized":"2024-02-14T17:17:43.844937+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:44:30.248401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:50.930986+0000","last_change":"2024-02-14T17:17:19.909427+0000","last_active":"2024-02-14T17:17:50.930986+0000","last_peered":"2024-02-14T17:17:50.930986+0000","last_clean":"2024-02-14T17:17:50.930986+0000","last_became_active":"2024-02-14T17:17:19.909084+0000","last_became_peered":"2024-02-14T17:17:19.909084+0000","last_unstale":"2024-02-14T17:17:50.930986+0000","last_undegraded":"2024-02-14T17:17:50.930986+0000","last_fullsized":"2024-02-14T17:17:50.930986+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:16.883480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842121+0000","last_change":"2024-02-14T17:17:15.884678+0000","last_active":"2024-02-14T17:17:43.842121+0000","last_peered":"2024-02-14T17:17:43.842121+0000","last_clean":"2024-02-14T17:17:43.842121+0000","last_became_active":"2024-02-14T17:17:15.883977+0000","last_became_peered":"2024-02-14T17:17:15.883977+0000","last_unstale":"2024-02-14T17:17:43.842121+0000","last_undegraded":"2024-02-14T17:17:43.842121+0000","last_fullsized":"2024-02-14T17:17:43.842121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:10:56.213069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842956+0000","last_change":"2024-02-14T17:17:17.897213+0000","last_active":"2024-02-14T17:17:43.842956+0000","last_peered":"2024-02-14T17:17:43.842956+0000","last_clean":"2024-02-14T17:17:43.842956+0000","last_became_active":"2024-02-14T17:17:17.896228+0000","last_became_peered":"2024-02-14T17:17:17.896228+0000","last_unstale":"2024-02-14T17:17:43.842956+0000","last_undegraded":"2024-02-14T17:17:43.842956+0000","last_fullsized":"2024-02-14T17:17:43.842956+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:07:27.384481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843608+0000","last_change":"2024-02-14T17:17:21.925909+0000","last_active":"2024-02-14T17:17:43.843608+0000","last_peered":"2024-02-14T17:17:43.843608+0000","last_clean":"2024-02-14T17:17:43.843608+0000","last_became_active":"2024-02-14T17:17:21.925586+0000","last_became_peered":"2024-02-14T17:17:21.925586+0000","last_unstale":"2024-02-14T17:17:43.843608+0000","last_undegraded":"2024-02-14T17:17:43.843608+0000","last_fullsized":"2024-02-14T17:17:43.843608+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:41:21.303093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845849+0000","last_change":"2024-02-14T17:17:19.911990+0000","last_active":"2024-02-14T17:17:43.845849+0000","last_peered":"2024-02-14T17:17:43.845849+0000","last_clean":"2024-02-14T17:17:43.845849+0000","last_became_active":"2024-02-14T17:17:19.911486+0000","last_became_peered":"2024-02-14T17:17:19.911486+0000","last_unstale":"2024-02-14T17:17:43.845849+0000","last_undegraded":"2024-02-14T17:17:43.845849+0000","last_fullsized":"2024-02-14T17:17:43.845849+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:32:59.643328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841749+0000","last_change":"2024-02-14T17:17:15.885145+0000","last_active":"2024-02-14T17:17:43.841749+0000","last_peered":"2024-02-14T17:17:43.841749+0000","last_clean":"2024-02-14T17:17:43.841749+0000","last_became_active":"2024-02-14T17:17:15.884189+0000","last_became_peered":"2024-02-14T17:17:15.884189+0000","last_unstale":"2024-02-14T17:17:43.841749+0000","last_undegraded":"2024-02-14T17:17:43.841749+0000","last_fullsized":"2024-02-14T17:17:43.841749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:10:48.334334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843264+0000","last_change":"2024-02-14T17:17:17.897930+0000","last_active":"2024-02-14T17:17:43.843264+0000","last_peered":"2024-02-14T17:17:43.843264+0000","last_clean":"2024-02-14T17:17:43.843264+0000","last_became_active":"2024-02-14T17:17:17.896426+0000","last_became_peered":"2024-02-14T17:17:17.896426+0000","last_unstale":"2024-02-14T17:17:43.843264+0000","last_undegraded":"2024-02-14T17:17:43.843264+0000","last_fullsized":"2024-02-14T17:17:43.843264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:28:39.643480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842993+0000","last_change":"2024-02-14T17:17:21.926041+0000","last_active":"2024-02-14T17:17:43.842993+0000","last_peered":"2024-02-14T17:17:43.842993+0000","last_clean":"2024-02-14T17:17:43.842993+0000","last_became_active":"2024-02-14T17:17:21.925745+0000","last_became_peered":"2024-02-14T17:17:21.925745+0000","last_unstale":"2024-02-14T17:17:43.842993+0000","last_undegraded":"2024-02-14T17:17:43.842993+0000","last_fullsized":"2024-02-14T17:17:43.842993+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:30:10.719144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844328+0000","last_change":"2024-02-14T17:17:15.886510+0000","last_active":"2024-02-14T17:17:43.844328+0000","last_peered":"2024-02-14T17:17:43.844328+0000","last_clean":"2024-02-14T17:17:43.844328+0000","last_became_active":"2024-02-14T17:17:15.885806+0000","last_became_peered":"2024-02-14T17:17:15.885806+0000","last_unstale":"2024-02-14T17:17:43.844328+0000","last_undegraded":"2024-02-14T17:17:43.844328+0000","last_fullsized":"2024-02-14T17:17:43.844328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:25:22.081089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843675+0000","last_change":"2024-02-14T17:17:17.894716+0000","last_active":"2024-02-14T17:17:43.843675+0000","last_peered":"2024-02-14T17:17:43.843675+0000","last_clean":"2024-02-14T17:17:43.843675+0000","last_became_active":"2024-02-14T17:17:17.894342+0000","last_became_peered":"2024-02-14T17:17:17.894342+0000","last_unstale":"2024-02-14T17:17:43.843675+0000","last_undegraded":"2024-02-14T17:17:43.843675+0000","last_fullsized":"2024-02-14T17:17:43.843675+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:56:10.807313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844010+0000","last_change":"2024-02-14T17:17:19.906769+0000","last_active":"2024-02-14T17:17:43.844010+0000","last_peered":"2024-02-14T17:17:43.844010+0000","last_clean":"2024-02-14T17:17:43.844010+0000","last_became_active":"2024-02-14T17:17:19.906418+0000","last_became_peered":"2024-02-14T17:17:19.906418+0000","last_unstale":"2024-02-14T17:17:43.844010+0000","last_undegraded":"2024-02-14T17:17:43.844010+0000","last_fullsized":"2024-02-14T17:17:43.844010+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:00:05.015696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844734+0000","last_change":"2024-02-14T17:17:21.926425+0000","last_active":"2024-02-14T17:17:43.844734+0000","last_peered":"2024-02-14T17:17:43.844734+0000","last_clean":"2024-02-14T17:17:43.844734+0000","last_became_active":"2024-02-14T17:17:21.926124+0000","last_became_peered":"2024-02-14T17:17:21.926124+0000","last_unstale":"2024-02-14T17:17:43.844734+0000","last_undegraded":"2024-02-14T17:17:43.844734+0000","last_fullsized":"2024-02-14T17:17:43.844734+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:39:01.307445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:50.930948+0000","last_change":"2024-02-14T17:17:19.909181+0000","last_active":"2024-02-14T17:17:50.930948+0000","last_peered":"2024-02-14T17:17:50.930948+0000","last_clean":"2024-02-14T17:17:50.930948+0000","last_became_active":"2024-02-14T17:17:19.908691+0000","last_became_peered":"2024-02-14T17:17:19.908691+0000","last_unstale":"2024-02-14T17:17:50.930948+0000","last_undegraded":"2024-02-14T17:17:50.930948+0000","last_fullsized":"2024-02-14T17:17:50.930948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:48:42.201337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845247+0000","last_change":"2024-02-14T17:17:15.887897+0000","last_active":"2024-02-14T17:17:43.845247+0000","last_peered":"2024-02-14T17:17:43.845247+0000","last_clean":"2024-02-14T17:17:43.845247+0000","last_became_active":"2024-02-14T17:17:15.887606+0000","last_became_peered":"2024-02-14T17:17:15.887606+0000","last_unstale":"2024-02-14T17:17:43.845247+0000","last_undegraded":"2024-02-14T17:17:43.845247+0000","last_fullsized":"2024-02-14T17:17:43.845247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:23.196204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843505+0000","last_change":"2024-02-14T17:17:17.892849+0000","last_active":"2024-02-14T17:17:43.843505+0000","last_peered":"2024-02-14T17:17:43.843505+0000","last_clean":"2024-02-14T17:17:43.843505+0000","last_became_active":"2024-02-14T17:17:17.891904+0000","last_became_peered":"2024-02-14T17:17:17.891904+0000","last_unstale":"2024-02-14T17:17:43.843505+0000","last_undegraded":"2024-02-14T17:17:43.843505+0000","last_fullsized":"2024-02-14T17:17:43.843505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:16:56.264422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843039+0000","last_change":"2024-02-14T17:17:21.926248+0000","last_active":"2024-02-14T17:17:43.843039+0000","last_peered":"2024-02-14T17:17:43.843039+0000","last_clean":"2024-02-14T17:17:43.843039+0000","last_became_active":"2024-02-14T17:17:21.925848+0000","last_became_peered":"2024-02-14T17:17:21.925848+0000","last_unstale":"2024-02-14T17:17:43.843039+0000","last_undegraded":"2024-02-14T17:17:43.843039+0000","last_fullsized":"2024-02-14T17:17:43.843039+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:23:37.998457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845507+0000","last_change":"2024-02-14T17:17:15.886505+0000","last_active":"2024-02-14T17:17:43.845507+0000","last_peered":"2024-02-14T17:17:43.845507+0000","last_clean":"2024-02-14T17:17:43.845507+0000","last_became_active":"2024-02-14T17:17:15.886136+0000","last_became_peered":"2024-02-14T17:17:15.886136+0000","last_unstale":"2024-02-14T17:17:43.845507+0000","last_undegraded":"2024-02-14T17:17:43.845507+0000","last_fullsized":"2024-02-14T17:17:43.845507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:03:52.304724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844959+0000","last_change":"2024-02-14T17:17:17.893380+0000","last_active":"2024-02-14T17:17:43.844959+0000","last_peered":"2024-02-14T17:17:43.844959+0000","last_clean":"2024-02-14T17:17:43.844959+0000","last_became_active":"2024-02-14T17:17:17.892442+0000","last_became_peered":"2024-02-14T17:17:17.892442+0000","last_unstale":"2024-02-14T17:17:43.844959+0000","last_undegraded":"2024-02-14T17:17:43.844959+0000","last_fullsized":"2024-02-14T17:17:43.844959+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:14:17.579346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842978+0000","last_change":"2024-02-14T17:17:19.906774+0000","last_active":"2024-02-14T17:17:43.842978+0000","last_peered":"2024-02-14T17:17:43.842978+0000","last_clean":"2024-02-14T17:17:43.842978+0000","last_became_active":"2024-02-14T17:17:19.906391+0000","last_became_peered":"2024-02-14T17:17:19.906391+0000","last_unstale":"2024-02-14T17:17:43.842978+0000","last_undegraded":"2024-02-14T17:17:43.842978+0000","last_fullsized":"2024-02-14T17:17:43.842978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:27:37.483011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.841711+0000","last_change":"2024-02-14T17:17:21.926162+0000","last_active":"2024-02-14T17:17:43.841711+0000","last_peered":"2024-02-14T17:17:43.841711+0000","last_clean":"2024-02-14T17:17:43.841711+0000","last_became_active":"2024-02-14T17:17:21.925850+0000","last_became_peered":"2024-02-14T17:17:21.925850+0000","last_unstale":"2024-02-14T17:17:43.841711+0000","last_undegraded":"2024-02-14T17:17:43.841711+0000","last_fullsized":"2024-02-14T17:17:43.841711+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:10.309431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843783+0000","last_change":"2024-02-14T17:17:15.889369+0000","last_active":"2024-02-14T17:17:43.843783+0000","last_peered":"2024-02-14T17:17:43.843783+0000","last_clean":"2024-02-14T17:17:43.843783+0000","last_became_active":"2024-02-14T17:17:15.888846+0000","last_became_peered":"2024-02-14T17:17:15.888846+0000","last_unstale":"2024-02-14T17:17:43.843783+0000","last_undegraded":"2024-02-14T17:17:43.843783+0000","last_fullsized":"2024-02-14T17:17:43.843783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:44:26.860854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844275+0000","last_change":"2024-02-14T17:17:17.893073+0000","last_active":"2024-02-14T17:17:43.844275+0000","last_peered":"2024-02-14T17:17:43.844275+0000","last_clean":"2024-02-14T17:17:43.844275+0000","last_became_active":"2024-02-14T17:17:17.892792+0000","last_became_peered":"2024-02-14T17:17:17.892792+0000","last_unstale":"2024-02-14T17:17:43.844275+0000","last_undegraded":"2024-02-14T17:17:43.844275+0000","last_fullsized":"2024-02-14T17:17:43.844275+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:06:53.719923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.843367+0000","last_change":"2024-02-14T17:17:19.910303+0000","last_active":"2024-02-14T17:17:43.843367+0000","last_peered":"2024-02-14T17:17:43.843367+0000","last_clean":"2024-02-14T17:17:43.843367+0000","last_became_active":"2024-02-14T17:17:19.909941+0000","last_became_peered":"2024-02-14T17:17:19.909941+0000","last_unstale":"2024-02-14T17:17:43.843367+0000","last_undegraded":"2024-02-14T17:17:43.843367+0000","last_fullsized":"2024-02-14T17:17:43.843367+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:23:13.437631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.842064+0000","last_change":"2024-02-14T17:17:21.920575+0000","last_active":"2024-02-14T17:17:43.842064+0000","last_peered":"2024-02-14T17:17:43.842064+0000","last_clean":"2024-02-14T17:17:43.842064+0000","last_became_active":"2024-02-14T17:17:21.920102+0000","last_became_peered":"2024-02-14T17:17:21.920102+0000","last_unstale":"2024-02-14T17:17:43.842064+0000","last_undegraded":"2024-02-14T17:17:43.842064+0000","last_fullsized":"2024-02-14T17:17:43.842064+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:20.903197+0000","last_clean_scrub_stamp":"2024-02-14T17:17:20.903197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:38:08.152329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844786+0000","last_change":"2024-02-14T17:17:15.884968+0000","last_active":"2024-02-14T17:17:43.844786+0000","last_peered":"2024-02-14T17:17:43.844786+0000","last_clean":"2024-02-14T17:17:43.844786+0000","last_became_active":"2024-02-14T17:17:15.884065+0000","last_became_peered":"2024-02-14T17:17:15.884065+0000","last_unstale":"2024-02-14T17:17:43.844786+0000","last_undegraded":"2024-02-14T17:17:43.844786+0000","last_fullsized":"2024-02-14T17:17:43.844786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:14.870540+0000","last_clean_scrub_stamp":"2024-02-14T17:17:14.870540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:17:31.935542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.845206+0000","last_change":"2024-02-14T17:17:17.895683+0000","last_active":"2024-02-14T17:17:43.845206+0000","last_peered":"2024-02-14T17:17:43.845206+0000","last_clean":"2024-02-14T17:17:43.845206+0000","last_became_active":"2024-02-14T17:17:17.895289+0000","last_became_peered":"2024-02-14T17:17:17.895289+0000","last_unstale":"2024-02-14T17:17:43.845206+0000","last_undegraded":"2024-02-14T17:17:43.845206+0000","last_fullsized":"2024-02-14T17:17:43.845206+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:16.875804+0000","last_clean_scrub_stamp":"2024-02-14T17:17:16.875804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:42:18.675102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-14T17:17:43.844023+0000","last_change":"2024-02-14T17:17:19.907060+0000","last_active":"2024-02-14T17:17:43.844023+0000","last_peered":"2024-02-14T17:17:43.844023+0000","last_clean":"2024-02-14T17:17:43.844023+0000","last_became_active":"2024-02-14T17:17:19.906748+0000","last_became_peered":"2024-02-14T17:17:19.906748+0000","last_unstale":"2024-02-14T17:17:43.844023+0000","last_undegraded":"2024-02-14T17:17:43.844023+0000","last_fullsized":"2024-02-14T17:17:43.844023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:17:18.889147+0000","last_clean_scrub_stamp":"2024-02-14T17:17:18.889147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:21:40.555140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738393,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 17:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.318}]},{"osd":2,"last update":"Wed Feb 14 17:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.35999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T17:18:02.473 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-14T17:18:02.474 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-14T17:18:02.474 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-14T17:18:02.474 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- ceph health --format=json 2024-02-14T17:18:02.714 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:18:03.909 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:03 smithi082 ceph-mon[25796]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:18:03.909 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:03 smithi082 ceph-mon[25796]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-14T17:18:04.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:03 smithi120 ceph-mon[30899]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:18:04.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:03 smithi120 ceph-mon[30899]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-14T17:18:04.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:03 smithi091 ceph-mon[30290]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:18:04.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:03 smithi091 ceph-mon[30290]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-14T17:18:04.722 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-14T17:18:04.722 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-14T17:18:04.982 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:04 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ::ffff:172.21.15.91 - - [14/Feb/2024:17:18:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:18:05.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:05 smithi082 ceph-mon[25796]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:05.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:05 smithi082 ceph-mon[25796]: from='client.? 172.21.15.82:0/4102613419' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T17:18:06.015 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-14T17:18:06.016 INFO:tasks.cephadm:Setup complete, yielding 2024-02-14T17:18:06.016 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-14T17:18:06.026 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-02-14T17:18:06.027 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-02-14T17:18:06.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:05 smithi120 ceph-mon[30899]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:06.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:05 smithi120 ceph-mon[30899]: from='client.? 172.21.15.82:0/4102613419' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T17:18:06.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:05 smithi091 ceph-mon[30290]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:06.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:05 smithi091 ceph-mon[30290]: from='client.? 172.21.15.82:0/4102613419' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T17:18:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:18:07.876 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:07 smithi082 ceph-mon[25796]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:08.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:07 smithi120 ceph-mon[30899]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:08.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:07 smithi091 ceph-mon[30290]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:09.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:09 smithi082 ceph-mon[25796]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:10.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:09 smithi120 ceph-mon[30899]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:10.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:09 smithi091 ceph-mon[30290]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:10.553 INFO:teuthology.orchestra.run.smithi082.stdout:167 167 2024-02-14T17:18:11.277 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- bash -c 'ceph orch status' 2024-02-14T17:18:11.525 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 podman[58137]: 2024-02-14 17:18:11.423213694 +0000 UTC m=+0.147535961 volume create 0c3ca61a108d8f9ce81a12deb4a8e8a02ce77c96f901081b1cb4d439867832f5 2024-02-14T17:18:11.525 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 podman[58137]: 2024-02-14 17:18:11.523593075 +0000 UTC m=+0.247915341 container create 003d103538210d4df03a77d1ebe013fc04670ae08966a9c45378908bea5f9b09 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:18:11.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:11 smithi082 ceph-mon[25796]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:12.007 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:18:12.055 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:11 smithi120 ceph-mon[30899]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:12.104 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:11 smithi091 ceph-mon[30290]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 podman[58137]: 2024-02-14 17:18:11.859511074 +0000 UTC m=+0.583833341 container init 003d103538210d4df03a77d1ebe013fc04670ae08966a9c45378908bea5f9b09 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 podman[58137]: 2024-02-14 17:18:11.865717885 +0000 UTC m=+0.590040152 container start 003d103538210d4df03a77d1ebe013fc04670ae08966a9c45378908bea5f9b09 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:11.877Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:11.877Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:11.878Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.82 port=9094 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:11.879Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:11.903Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:11.903Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:11.906Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:11.906Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 bash[58137]: 003d103538210d4df03a77d1ebe013fc04670ae08966a9c45378908bea5f9b09 2024-02-14T17:18:12.131 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:11 smithi082 systemd[1]: Started Ceph alertmanager.a for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:18:12.413 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:12 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:12] ENGINE Bus STOPPING 2024-02-14T17:18:12.414 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:12 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:18:12.414 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:12 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:12] ENGINE Bus STOPPED 2024-02-14T17:18:12.414 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:12 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:12] ENGINE Bus STARTING 2024-02-14T17:18:12.414 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:12 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:12] ENGINE Serving on http://:::9283 2024-02-14T17:18:12.414 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:12 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:12] ENGINE Bus STARTED 2024-02-14T17:18:13.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:13.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.142 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-14T17:18:13.142 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.142 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.142 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:18:13.142 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:18:13.142 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.142 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-mon[25796]: Deploying daemon grafana.a on smithi091 2024-02-14T17:18:13.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:13.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-14T17:18:13.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:18:13.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:18:13.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.306 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:13 smithi120 ceph-mon[30899]: Deploying daemon grafana.a on smithi091 2024-02-14T17:18:13.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:13.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-14T17:18:13.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.355 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.355 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:18:13.355 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:18:13.355 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:13.355 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:13 smithi091 ceph-mon[30290]: Deploying daemon grafana.a on smithi091 2024-02-14T17:18:13.888 INFO:teuthology.orchestra.run.smithi082.stdout:Backend: cephadm 2024-02-14T17:18:13.888 INFO:teuthology.orchestra.run.smithi082.stdout:Available: Yes 2024-02-14T17:18:13.888 INFO:teuthology.orchestra.run.smithi082.stdout:Paused: No 2024-02-14T17:18:14.189 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:13 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:13.880Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00025916s 2024-02-14T17:18:14.475 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-14T17:18:14.712 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:18:15.100 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:14 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ::ffff:172.21.15.91 - - [14/Feb/2024:17:18:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:18:15.100 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:14 smithi082 ceph-mon[25796]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:15.100 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:14 smithi082 ceph-mon[25796]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:15.100 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:14 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:15.100 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:14 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:18:15.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:14 smithi091 ceph-mon[30290]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:15.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:14 smithi091 ceph-mon[30290]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:15.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:14 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:15.103 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:14 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:18:15.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:14 smithi120 ceph-mon[30899]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:15.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:14 smithi120 ceph-mon[30899]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:15.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:14 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' 2024-02-14T17:18:15.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:14 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:alertmanager.a smithi082 *:9093,9094 starting - - - - 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.x smithi091 *:8443,8765 running (2m) 30s ago 2m 448M - 19.0.0-1430-g8727ce93 170846cf6432 e217a9aaa51d 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.y smithi082 *:9283,8765 running (4m) 24s ago 4m 477M - 19.0.0-1430-g8727ce93 170846cf6432 ff02decca377 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.z smithi120 *:8443,8765 running (2m) 31s ago 2m 447M - 19.0.0-1430-g8727ce93 170846cf6432 c9754de8f5c2 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:mon.a smithi082 running (4m) 24s ago 4m 52.3M 2048M 19.0.0-1430-g8727ce93 170846cf6432 9549c93bb8c4 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:mon.b smithi091 running (2m) 30s ago 2m 43.8M 2048M 19.0.0-1430-g8727ce93 170846cf6432 d86e71bb3cb0 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:mon.c smithi120 running (2m) 31s ago 2m 46.0M 2048M 19.0.0-1430-g8727ce93 170846cf6432 40bbe8467118 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 smithi082 running (2m) 24s ago 2m 58.0M 15.6G 19.0.0-1430-g8727ce93 170846cf6432 9aab06971476 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 smithi091 running (108s) 30s ago 107s 51.5M 15.6G 19.0.0-1430-g8727ce93 170846cf6432 61f011004186 2024-02-14T17:18:16.579 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 smithi120 running (84s) 31s ago 83s 59.4M 16.6G 19.0.0-1430-g8727ce93 170846cf6432 e9f6b24ab5aa 2024-02-14T17:18:16.580 INFO:teuthology.orchestra.run.smithi082.stdout:prometheus.a smithi091 *:9095 running (42s) 30s ago 40s 26.7M - 2.43.0 a07b618ecd1d 829071073b81 2024-02-14T17:18:16.580 INFO:teuthology.orchestra.run.smithi082.stdout:rgw.foo.a smithi082 *:80 running (62s) 24s ago 61s 75.6M - 19.0.0-1430-g8727ce93 170846cf6432 52f0cd3e4f0d 2024-02-14T17:18:17.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:16 smithi082 ceph-mon[25796]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:16 smithi082 ceph-mon[25796]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:17.265 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-14T17:18:17.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:16 smithi120 ceph-mon[30899]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:17.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:16 smithi120 ceph-mon[30899]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:17.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:16 smithi091 ceph-mon[30290]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:17.354 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:16 smithi091 ceph-mon[30290]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:18:18.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:18 smithi082 ceph-mon[25796]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:19.265 INFO:teuthology.orchestra.run.smithi082.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-14T17:18:19.265 INFO:teuthology.orchestra.run.smithi082.stdout:alertmanager ?:9093,9094 0/1 - 60s smithi082=a;count:1 2024-02-14T17:18:19.265 INFO:teuthology.orchestra.run.smithi082.stdout:grafana ?:3000 0/1 - 57s smithi091=a;count:1 2024-02-14T17:18:19.265 INFO:teuthology.orchestra.run.smithi082.stdout:mgr 3/3 33s ago 2m smithi082=y;smithi091=x;smithi120=z;count:3 2024-02-14T17:18:19.265 INFO:teuthology.orchestra.run.smithi082.stdout:mon 3/3 33s ago 3m smithi082:172.21.15.82=a;smithi091:172.21.15.91=b;smithi120:172.21.15.120=c;count:3 2024-02-14T17:18:19.266 INFO:teuthology.orchestra.run.smithi082.stdout:node-exporter ?:9100 0/3 - 63s smithi082=a;smithi091=b;smithi120=c;count:3 2024-02-14T17:18:19.266 INFO:teuthology.orchestra.run.smithi082.stdout:osd 3 33s ago - 2024-02-14T17:18:19.266 INFO:teuthology.orchestra.run.smithi082.stdout:prometheus ?:9095 1/1 33s ago 65s smithi091=a;count:1 2024-02-14T17:18:19.266 INFO:teuthology.orchestra.run.smithi082.stdout:rgw.foo.a ?:80 1/1 27s ago 64s smithi082=foo.a;count:1 2024-02-14T17:18:19.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:18 smithi120 ceph-mon[30899]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:19.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:18 smithi091 ceph-mon[30290]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:19.859 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-14T17:18:20.095 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:18:20.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:19 smithi082 ceph-mon[25796]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:20.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:19 smithi120 ceph-mon[30899]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:20.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:19 smithi091 ceph-mon[30290]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:21.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:20 smithi082 ceph-mon[25796]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:21.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:20 smithi120 ceph-mon[30899]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:21.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:20 smithi091 ceph-mon[30290]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:HOST ADDR LABELS STATUS 2024-02-14T17:18:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 172.21.15.82 2024-02-14T17:18:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:smithi091 172.21.15.91 2024-02-14T17:18:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:smithi120 172.21.15.120 2024-02-14T17:18:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:3 hosts in cluster 2024-02-14T17:18:22.231 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:18:21 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:18:21.881Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001619491s 2024-02-14T17:18:22.714 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-14T17:18:22.950 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:18:23.201 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:22 smithi082 ceph-mon[25796]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:23.201 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:22 smithi082 ceph-mon[25796]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:23.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:22 smithi120 ceph-mon[30899]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:23.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:22 smithi120 ceph-mon[30899]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:23.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:22 smithi091 ceph-mon[30290]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:23.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:22 smithi091 ceph-mon[30290]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:24.863 INFO:teuthology.orchestra.run.smithi082.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-14T17:18:24.863 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-14T17:18:24.863 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme1n1 ssd Linux_4f764867856cc8ce0cd6 89.4G Yes 2m ago 2024-02-14T17:18:24.863 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme2n1 ssd Linux_364aff1b9151f94e5324 89.4G Yes 2m ago 2024-02-14T17:18:24.863 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme3n1 ssd Linux_8618400abe87701ac7cc 89.4G Yes 2m ago 2024-02-14T17:18:24.863 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme4n1 ssd Linux_6a816038c081d050ccd8 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-14T17:18:24.863 INFO:teuthology.orchestra.run.smithi082.stdout:smithi091 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300AG400BGN 372G No 108s ago Has a FileSystem, LVM detected 2024-02-14T17:18:24.863 INFO:teuthology.orchestra.run.smithi082.stdout:smithi091 /dev/nvme1n1 ssd Linux_6bcc575aa7cf1969d39d 89.4G Yes 108s ago 2024-02-14T17:18:24.864 INFO:teuthology.orchestra.run.smithi082.stdout:smithi091 /dev/nvme2n1 ssd Linux_cf342decd805f61a2b68 89.4G Yes 108s ago 2024-02-14T17:18:24.864 INFO:teuthology.orchestra.run.smithi082.stdout:smithi091 /dev/nvme3n1 ssd Linux_2237913d6ea39ee4b780 89.4G Yes 108s ago 2024-02-14T17:18:24.864 INFO:teuthology.orchestra.run.smithi082.stdout:smithi091 /dev/nvme4n1 ssd Linux_deadadc57fa37af37191 89.4G No 108s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-14T17:18:24.864 INFO:teuthology.orchestra.run.smithi082.stdout:smithi120 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XV400BGN 372G No 84s ago Has a FileSystem, LVM detected 2024-02-14T17:18:24.864 INFO:teuthology.orchestra.run.smithi082.stdout:smithi120 /dev/nvme1n1 ssd Linux_c7437a01a5cab3a8df82 89.4G Yes 84s ago 2024-02-14T17:18:24.864 INFO:teuthology.orchestra.run.smithi082.stdout:smithi120 /dev/nvme2n1 ssd Linux_5175f937f3136689a646 89.4G Yes 84s ago 2024-02-14T17:18:24.864 INFO:teuthology.orchestra.run.smithi082.stdout:smithi120 /dev/nvme3n1 ssd Linux_de5d6bd4677e9abf306b 89.4G Yes 84s ago 2024-02-14T17:18:24.864 INFO:teuthology.orchestra.run.smithi082.stdout:smithi120 /dev/nvme4n1 ssd Linux_08f82a3d3cdd0c3b1bd2 89.4G No 84s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-14T17:18:25.173 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:24 smithi082 ceph-mon[25796]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:25.173 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:24 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ::ffff:172.21.15.91 - - [14/Feb/2024:17:18:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:18:25.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:24 smithi120 ceph-mon[30899]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:25.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:24 smithi091 ceph-mon[30290]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:25.569 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ls --format yaml' 2024-02-14T17:18:25.807 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:18:26.184 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:25 smithi082 ceph-mon[25796]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:26.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:25 smithi120 ceph-mon[30899]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:26.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:25 smithi091 ceph-mon[30290]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:27.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:26 smithi082 ceph-mon[25796]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:27.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:26 smithi120 ceph-mon[30899]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:27.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:26 smithi091 ceph-mon[30290]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:27.640 INFO:teuthology.orchestra.run.smithi082.stdout:service_type: alertmanager 2024-02-14T17:18:27.640 INFO:teuthology.orchestra.run.smithi082.stdout:service_name: alertmanager 2024-02-14T17:18:27.640 INFO:teuthology.orchestra.run.smithi082.stdout:placement: 2024-02-14T17:18:27.640 INFO:teuthology.orchestra.run.smithi082.stdout: count: 1 2024-02-14T17:18:27.640 INFO:teuthology.orchestra.run.smithi082.stdout: hosts: 2024-02-14T17:18:27.640 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi082=a 2024-02-14T17:18:27.640 INFO:teuthology.orchestra.run.smithi082.stdout:status: 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout: created: '2024-02-14T17:17:18.942178Z' 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout: ports: 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout: - 9093 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout: - 9094 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout: running: 0 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout: size: 1 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout:events: 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout:- 2024-02-14T17:18:12.083254Z service:alertmanager [INFO] "service was created" 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout:--- 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout:service_type: grafana 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout:service_name: grafana 2024-02-14T17:18:27.641 INFO:teuthology.orchestra.run.smithi082.stdout:placement: 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: count: 1 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: hosts: 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi091=a 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout:spec: 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: anonymous_access: true 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: protocol: https 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout:status: 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: created: '2024-02-14T17:17:21.574125Z' 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: ports: 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: - 3000 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: running: 0 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout: size: 1 2024-02-14T17:18:27.642 INFO:teuthology.orchestra.run.smithi082.stdout:--- 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout:service_type: mgr 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout:service_name: mgr 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout:placement: 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout: count: 3 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout: hosts: 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi082=y 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi091=x 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi120=z 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout:status: 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout: created: '2024-02-14T17:15:40.752074Z' 2024-02-14T17:18:27.643 INFO:teuthology.orchestra.run.smithi082.stdout: last_refresh: '2024-02-14T17:17:45.555429Z' 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout: running: 3 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout: size: 3 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout:--- 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout:service_type: mon 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout:service_name: mon 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout:placement: 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout: count: 3 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout: hosts: 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi082:172.21.15.82=a 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi091:172.21.15.91=b 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi120:172.21.15.120=c 2024-02-14T17:18:27.644 INFO:teuthology.orchestra.run.smithi082.stdout:status: 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: created: '2024-02-14T17:15:11.720910Z' 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: last_refresh: '2024-02-14T17:17:45.555312Z' 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: running: 3 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: size: 3 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout:--- 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout:service_type: node-exporter 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout:service_name: node-exporter 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout:placement: 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: count: 3 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: hosts: 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi082=a 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi091=b 2024-02-14T17:18:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi120=c 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout:status: 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout: created: '2024-02-14T17:17:16.233279Z' 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout: ports: 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout: - 9100 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout: running: 0 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout: size: 3 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout:--- 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout:service_type: osd 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout:service_name: osd 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout:unmanaged: true 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout:spec: 2024-02-14T17:18:27.646 INFO:teuthology.orchestra.run.smithi082.stdout: filter_logic: AND 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout: objectstore: bluestore 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout:status: 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout: container_image_id: 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout: last_refresh: '2024-02-14T17:17:45.555504Z' 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout: running: 3 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout: size: 3 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout:--- 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout:service_type: prometheus 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout:service_name: prometheus 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout:placement: 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout: count: 1 2024-02-14T17:18:27.647 INFO:teuthology.orchestra.run.smithi082.stdout: hosts: 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi091=a 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout:status: 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout: created: '2024-02-14T17:17:13.645109Z' 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout: last_refresh: '2024-02-14T17:17:46.251129Z' 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout: ports: 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout: - 9095 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout: running: 1 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout: size: 1 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout:--- 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout:service_type: rgw 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout:service_id: foo.a 2024-02-14T17:18:27.648 INFO:teuthology.orchestra.run.smithi082.stdout:service_name: rgw.foo.a 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout:placement: 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout: count: 1 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout: hosts: 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout: - smithi082=foo.a 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout:status: 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout: created: '2024-02-14T17:17:14.854000Z' 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout: last_refresh: '2024-02-14T17:17:52.072395Z' 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout: ports: 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout: - 80 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout: running: 1 2024-02-14T17:18:27.649 INFO:teuthology.orchestra.run.smithi082.stdout: size: 1 2024-02-14T17:18:28.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:27 smithi082 ceph-mon[25796]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-14T17:18:28.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:27 smithi120 ceph-mon[30899]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-14T17:18:28.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:27 smithi091 ceph-mon[30290]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-14T17:18:28.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-02-14T17:18:28.641 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:18:29.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:28 smithi082 ceph-mon[25796]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:29.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:28 smithi120 ceph-mon[30899]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:29.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:28 smithi091 ceph-mon[30290]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:30.103 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:29 smithi082 ceph-mon[25796]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:18:30.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:29 smithi120 ceph-mon[30899]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:18:30.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:29 smithi091 ceph-mon[30290]: from='mgr.14445 172.21.15.82:0/1784259086' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:18:30.549 INFO:teuthology.orchestra.run.smithi082.stdout:osd 3 44s ago - 2024-02-14T17:18:31.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:30 smithi082 ceph-mon[25796]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:31.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:30 smithi082 ceph-mon[25796]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:31.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:30 smithi120 ceph-mon[30899]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:31.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:30 smithi120 ceph-mon[30899]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:31.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:30 smithi091 ceph-mon[30290]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:18:31.353 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:30 smithi091 ceph-mon[30290]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:18:31.514 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-14T17:18:31.523 INFO:tasks.cephadm:Teardown begin 2024-02-14T17:18:31.524 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:18:31.554 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:18:31.585 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:18:31.620 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-14T17:18:31.620 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T17:18:31.636 DEBUG:teuthology.orchestra.run.smithi091:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T17:18:31.651 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T17:18:31.676 INFO:tasks.cephadm:Stopping all daemons... 2024-02-14T17:18:31.677 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-14T17:18:31.677 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.a 2024-02-14T17:18:31.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:31 smithi082 systemd[1]: Stopping Ceph mon.a for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:18:32.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:32 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a[25773]: 2024-02-14T17:18:32.095+0000 7fa0a1f5a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:18:32.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:32 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a[25773]: 2024-02-14T17:18:32.095+0000 7fa0a1f5a700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-14T17:18:32.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:32 smithi082 podman[59757]: 2024-02-14 17:18:32.331633278 +0000 UTC m=+0.466751618 container died 9549c93bb8c407dffc00098a9d7a8f6db94d09ae39ce52ea96d8b4860cb5433b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T17:18:32.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:32 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:32] ENGINE Bus STOPPING 2024-02-14T17:18:32.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:32 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:32] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:18:32.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:32 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:32] ENGINE Bus STOPPED 2024-02-14T17:18:32.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:32 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:32] ENGINE Bus STARTING 2024-02-14T17:18:32.482 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:32 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:32] ENGINE Serving on http://:::9283 2024-02-14T17:18:32.483 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:32 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: [14/Feb/2024:17:18:32] ENGINE Bus STARTED 2024-02-14T17:18:32.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:32 smithi082 podman[59757]: 2024-02-14 17:18:32.709799199 +0000 UTC m=+0.844917519 container cleanup 9549c93bb8c407dffc00098a9d7a8f6db94d09ae39ce52ea96d8b4860cb5433b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-14T17:18:32.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Feb 14 17:18:32 smithi082 bash[59757]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-a 2024-02-14T17:18:33.120 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.a.service' 2024-02-14T17:18:33.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:18:33.166 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-14T17:18:33.167 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-14T17:18:33.167 DEBUG:teuthology.orchestra.run.smithi091:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.b 2024-02-14T17:18:33.549 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:33 smithi091 systemd[1]: Stopping Ceph mon.b for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:18:33.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:33 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-b[30267]: 2024-02-14T17:18:33.547+0000 7f2f02b59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:18:33.853 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:33 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-b[30267]: 2024-02-14T17:18:33.547+0000 7f2f02b59700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-14T17:18:33.854 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:33 smithi091 podman[42637]: 2024-02-14 17:18:33.721270022 +0000 UTC m=+0.325286677 container died d86e71bb3cb0698af58deaafbff0ae095af9ed592fad252411a13b659b4baca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-b, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-02-14T17:18:34.592 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:34 smithi091 podman[42637]: 2024-02-14 17:18:34.247902852 +0000 UTC m=+0.851919513 container cleanup d86e71bb3cb0698af58deaafbff0ae095af9ed592fad252411a13b659b4baca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-b, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-14T17:18:34.592 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:34 smithi091 bash[42637]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-b 2024-02-14T17:18:34.592 INFO:journalctl@ceph.mon.b.smithi091.stdout:Feb 14 17:18:34 smithi091 podman[42656]: 2024-02-14 17:18:34.424459322 +0000 UTC m=+0.704465400 container remove d86e71bb3cb0698af58deaafbff0ae095af9ed592fad252411a13b659b4baca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214) 2024-02-14T17:18:35.231 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:34 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: ::ffff:172.21.15.91 - - [14/Feb/2024:17:18:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:18:35.537 DEBUG:teuthology.orchestra.run.smithi091:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.b.service' 2024-02-14T17:18:35.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:18:35.595 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-14T17:18:35.595 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-14T17:18:35.595 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.c 2024-02-14T17:18:35.974 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:35 smithi120 systemd[1]: Stopping Ceph mon.c for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:18:36.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:35 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-c[30876]: 2024-02-14T17:18:35.972+0000 7f397268f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:18:36.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:35 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-c[30876]: 2024-02-14T17:18:35.972+0000 7f397268f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-14T17:18:36.305 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:36 smithi120 podman[44056]: 2024-02-14 17:18:36.136583099 +0000 UTC m=+0.345497389 container died 40bbe8467118cf800f0b81455edfb8d0a2e1fe7549c27195ca3ff39ef1e139c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T17:18:36.755 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:36 smithi120 podman[44056]: 2024-02-14 17:18:36.578052717 +0000 UTC m=+0.786967007 container cleanup 40bbe8467118cf800f0b81455edfb8d0a2e1fe7549c27195ca3ff39ef1e139c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-14T17:18:36.755 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:36 smithi120 bash[44056]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-c 2024-02-14T17:18:37.001 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.c.service' 2024-02-14T17:18:37.025 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:36 smithi120 podman[44075]: 2024-02-14 17:18:36.754269086 +0000 UTC m=+0.616499035 container remove 40bbe8467118cf800f0b81455edfb8d0a2e1fe7549c27195ca3ff39ef1e139c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-14T17:18:37.025 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:36 smithi120 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.c.service: Deactivated successfully. 2024-02-14T17:18:37.025 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:36 smithi120 systemd[1]: Stopped Ceph mon.c for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:18:37.025 INFO:journalctl@ceph.mon.c.smithi120.stdout:Feb 14 17:18:36 smithi120 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mon.c.service: Consumed 3.547s CPU time. 2024-02-14T17:18:37.046 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:18:37.046 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-14T17:18:37.046 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-14T17:18:37.046 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.y 2024-02-14T17:18:37.260 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 podman[42947]: 2024-02-14 17:18:37.154316278 +0000 UTC m=+0.121047847 container create 403d55489ae7a955cf6bf062151dba7ffccdbaa79a8b2ba017c014c2bd5e8be6 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a, vcs-type=git, description=Ceph Grafana Container, io.buildah.version=1.32.0, version=8.5, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , release=239.1651231664, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-02-14T17:18:37.350 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:37 smithi082 systemd[1]: Stopping Ceph mgr.y for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:18:37.609 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 podman[42947]: 2024-02-14 17:18:37.356457991 +0000 UTC m=+0.323189568 container init 403d55489ae7a955cf6bf062151dba7ffccdbaa79a8b2ba017c014c2bd5e8be6 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, release=239.1651231664, io.buildah.version=1.32.0, version=8.5, vcs-type=git, description=Ceph Grafana Container, name=ubi8, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.expose-services=) 2024-02-14T17:18:37.609 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 podman[42947]: 2024-02-14 17:18:37.361852239 +0000 UTC m=+0.328583811 container start 403d55489ae7a955cf6bf062151dba7ffccdbaa79a8b2ba017c014c2bd5e8be6 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a, maintainer=Paul Cuzner , architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, name=ubi8, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, release=239.1651231664, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, vendor=Red Hat, Inc., vcs-type=git) 2024-02-14T17:18:37.609 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 bash[42947]: 403d55489ae7a955cf6bf062151dba7ffccdbaa79a8b2ba017c014c2bd5e8be6 2024-02-14T17:18:37.609 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 systemd[1]: Started Ceph grafana.a for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:18:37.609 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508683426Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-14T17:18:37.609 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508902016Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-14T17:18:37.610 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508920706Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-14T17:18:37.610 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508928613Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-14T17:18:37.610 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508934646Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-14T17:18:37.610 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508940625Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-14T17:18:37.610 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508946912Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-14T17:18:37.610 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508952752Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508959354Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508970883Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508977011Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.508983025Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.50900266Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=settings t=2024-02-14T17:18:37.509009853Z level=info msg="App mode production" 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=sqlstore t=2024-02-14T17:18:37.509058439Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=sqlstore t=2024-02-14T17:18:37.509080812Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.509951641Z level=info msg="Starting DB migrations" 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.510038566Z level=info msg="Executing migration" id="create migration_log table" 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.510593826Z level=info msg="Executing migration" id="create user table" 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.511044612Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-14T17:18:37.611 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.511492989Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-14T17:18:37.612 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.511931523Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-14T17:18:37.612 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.512545395Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-14T17:18:37.612 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.512925529Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-14T17:18:37.612 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.513626011Z level=info msg="Executing migration" id="create user table v2" 2024-02-14T17:18:37.612 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.514042118Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-14T17:18:37.612 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.514428442Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-14T17:18:37.612 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.514867053Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-14T17:18:37.612 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.515181795Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-14T17:18:37.612 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.515556175Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.515989596Z level=info msg="Executing migration" id="Update user table charset" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.516277599Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.516700643Z level=info msg="Executing migration" id="Add missing user data" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.517061345Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.517504315Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.517906149Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.518325074Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.519459607Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.519886257Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.520329885Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.520729075Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.521717201Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.522238182Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.523228685Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-14T17:18:37.613 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.524018439Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.524644953Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.525584648Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.526351114Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.527241434Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.527666003Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.528571659Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.529083986Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.529509825Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.530281695Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.530629096Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.531475239Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.531851018Z level=info msg="Executing migration" id="create star table" 2024-02-14T17:18:37.614 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.532318616Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-14T17:18:37.615 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.532767318Z level=info msg="Executing migration" id="create org table v1" 2024-02-14T17:18:37.615 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.533234103Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-14T17:18:37.615 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.534164441Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-14T17:18:37.615 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.534912878Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-14T17:18:37.615 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.535904772Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-14T17:18:37.615 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.536676555Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-14T17:18:37.616 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.537507242Z level=info msg="Executing migration" id="Update org table charset" 2024-02-14T17:18:37.616 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.538366413Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-14T17:18:37.616 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.53925572Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-14T17:18:37.616 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.540232208Z level=info msg="Executing migration" id="create dashboard table" 2024-02-14T17:18:37.616 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.540921353Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-14T17:18:37.616 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.54183359Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-14T17:18:37.616 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.542688359Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-14T17:18:37.616 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.543567816Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-14T17:18:37.616 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.544392167Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.54539184Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.546276835Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.547157393Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.548008721Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.548905747Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.549749491Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.550595207Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.551542612Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.554331265Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.557218831Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.560054678Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.562904Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.56571816Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-14T17:18:37.617 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.568535856Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.571411929Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.574132129Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.577247666Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.580101503Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.583017919Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.5858893Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.588800713Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.591885625Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.594689887Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.597491829Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.600328726Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.603031505Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.605802009Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-14T17:18:37.618 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.608614029Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-14T17:18:37.619 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.611221823Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-14T17:18:37.656 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:37 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:18:37.348+0000 7fe6e9ce4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:18:37.656 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:37 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y[26056]: 2024-02-14T17:18:37.348+0000 7fe6e9ce4700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-14T17:18:37.656 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:37 smithi082 podman[59900]: 2024-02-14 17:18:37.380746733 +0000 UTC m=+0.140729185 container died ff02decca377922d493ced8757908497b9b9c41fff3e973b00cb7cf31ad517fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:18:37.860 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.61388464Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.616608424Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.619413861Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.622249946Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.624891107Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.627671582Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.630409329Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.633056915Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.635866214Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.638712441Z level=info msg="Executing migration" id="create data_source table" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.641506285Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.644243825Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.646799723Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-14T17:18:37.861 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.649483857Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-14T17:18:37.862 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.650404144Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-14T17:18:37.862 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.65152416Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-14T17:18:37.862 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.652442321Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-14T17:18:37.862 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.653203646Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-14T17:18:37.862 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.654181766Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-14T17:18:37.863 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.654922202Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-14T17:18:37.863 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.65577463Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-14T17:18:37.863 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.656611624Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-14T17:18:37.863 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.657579294Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-14T17:18:37.863 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.65830921Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-14T17:18:37.863 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.659150181Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-14T17:18:37.863 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.660105596Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-14T17:18:37.863 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.660927127Z level=info msg="Executing migration" id="Add uid column" 2024-02-14T17:18:37.863 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.66178842Z level=info msg="Executing migration" id="Update uid value" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.66274681Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.663461215Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.664319845Z level=info msg="Executing migration" id="create api_key table" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.665190055Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.666036155Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.666931451Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.667780738Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.668741065Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.669203898Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.670223389Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.671336183Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.67228867Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-14T17:18:37.864 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.672748244Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.673714815Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.674535848Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.675277821Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.675667383Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.676474152Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.677088966Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.677673145Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.678613757Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.679284904Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.682327041Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.684954557Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.687710894Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.688195447Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.691125229Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-14T17:18:37.865 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.694135009Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.697065668Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.697541475Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.700293999Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.700879155Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.703264933Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.706172132Z level=info msg="Executing migration" id="create quota table v1" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.706571289Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.709469943Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.712245117Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.715034872Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.715643583Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.718338572Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.718645092Z level=info msg="Executing migration" id="create session table" 2024-02-14T17:18:37.866 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.721177527Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.723853772Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.724199231Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.726961865Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.729892253Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.732459844Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.732751303Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.735453959Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.735862842Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.73830229Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.740857473Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.741533816Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.741862865Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.742480429Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.743106032Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-14T17:18:37.867 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.743419045Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.743849117Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.744452232Z level=info msg="Executing migration" id="create alert table v1" 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.747235378Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.747776645Z level=info msg="Executing migration" id="add index alert state" 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.750593619Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.751065443Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.751449711Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.751917438Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.752368263Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-14T17:18:37.868 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.755287214Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.755758567Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.756182511Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.75651616Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.756858021Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.757234311Z level=info msg="Executing migration" id="Add column is_default" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.757894665Z level=info msg="Executing migration" id="Add column frequency" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.758533775Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.759168798Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.75980453Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.760239048Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.760538495Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.760816105Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-14T17:18:37.869 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.761218107Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.761651737Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.762015808Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.762439187Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.764930712Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.765604013Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.768530007Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.771300785Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.773937481Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.774388609Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.777125282Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.779972258Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.782640323Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-14T17:18:37.870 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.7854383Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.78798535Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.790783974Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.793284596Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.793723562Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.796440997Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.799304219Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.801851505Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.802173357Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.805035138Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.807661957Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.810405991Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.810870265Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-14T17:18:37.871 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.811776649Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-14T17:18:37.872 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.812573846Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-14T17:18:37.872 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.814254661Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-14T17:18:37.872 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.814643358Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-14T17:18:37.872 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.815636558Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-14T17:18:37.872 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.816357036Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-14T17:18:37.872 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.817344484Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.817778679Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.818610865Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.819458731Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.820385524Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.820989997Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.821805548Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.822529531Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.822992706Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.823920168Z level=info msg="Executing migration" id="Move region to single row" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.824374493Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.82480683Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.825651909Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.826098052Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-14T17:18:37.873 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.826538781Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.827382298Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.827831274Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.828750232Z level=info msg="Executing migration" id="create test_data table" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.829270182Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.829720577Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.830526947Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.830969936Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.831808078Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.832132745Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.83270176Z level=info msg="Executing migration" id="create team table" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.833638433Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.834078456Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.834953252Z level=info msg="Executing migration" id="create team member table" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.835936521Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-14T17:18:37.874 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.8387171Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.839181768Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.841978863Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.844986488Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.847813276Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.84872262Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.851485573Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.85412167Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.854580994Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.857024524Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.859875843Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.862686129Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.863262855Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.863742934Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-14T17:18:37.875 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.864135876Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-14T17:18:37.981 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:37 smithi082 podman[59900]: 2024-02-14 17:18:37.754592479 +0000 UTC m=+0.514574932 container cleanup ff02decca377922d493ced8757908497b9b9c41fff3e973b00cb7cf31ad517fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T17:18:37.982 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Feb 14 17:18:37 smithi082 bash[59900]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-y 2024-02-14T17:18:38.111 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.864492738Z level=info msg="Executing migration" id="create tag table" 2024-02-14T17:18:38.111 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.864890626Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-14T17:18:38.111 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.865324617Z level=info msg="Executing migration" id="create login attempt table" 2024-02-14T17:18:38.111 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.865724197Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-14T17:18:38.111 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.866161238Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-14T17:18:38.111 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.866599088Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-14T17:18:38.112 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.868495646Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-14T17:18:38.112 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.868891317Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-14T17:18:38.112 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.86930165Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-14T17:18:38.112 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.869618681Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-14T17:18:38.112 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.869941795Z level=info msg="Executing migration" id="create user auth table" 2024-02-14T17:18:38.112 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.870289488Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-14T17:18:38.115 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.870700047Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-14T17:18:38.115 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.870956567Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-14T17:18:38.115 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.871706296Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.872452282Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.873224266Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.87398489Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.874414874Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.875132348Z level=info msg="Executing migration" id="create server_lock table" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.875529282Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.875953811Z level=info msg="Executing migration" id="create user auth token table" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.876377382Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.879149482Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.879586659Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.880002724Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-14T17:18:38.116 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.880761754Z level=info msg="Executing migration" id="create cache_data table" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.881155763Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.881586293Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.881992911Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.884907581Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.885241488Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.88555395Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.885970387Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.888777819Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.891696452Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.892050974Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-14T17:18:38.117 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.894370962Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-14T17:18:38.119 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.895253265Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-14T17:18:38.119 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.897943947Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-14T17:18:38.119 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.900491149Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-14T17:18:38.119 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.901342465Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-14T17:18:38.119 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.902192597Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-14T17:18:38.119 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.903041022Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-14T17:18:38.119 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.903900821Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-14T17:18:38.119 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.904409066Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-14T17:18:38.119 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.905235673Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.906102363Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.907004914Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.907708686Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.908525566Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.909013256Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.909816512Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.910745494Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.911386582Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.914809706Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.918151225Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.919075787Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.919648599Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.920568213Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-14T17:18:38.120 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.921502548Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.922240723Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.922663781Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.923606237Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.924380545Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.925236731Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.92605095Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.926931588Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.927890111Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.928625458Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.929421363Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.930266511Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.931261319Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.932164756Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.932984106Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-14T17:18:38.121 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.933807717Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.934287229Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.936621911Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.939690772Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.940017106Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.94298559Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.94574069Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.948619614Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.94964984Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.952704129Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.955473035Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.958348101Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.959178435Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.961949797Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.965005107Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-14T17:18:38.122 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.967737376Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.968668706Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.971436813Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.974395839Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.977044175Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.977493473Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.980253191Z level=info msg="Executing migration" id="create alert_image table" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.982948989Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.98560765Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.985925444Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.988795318Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.99154152Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.991641243Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-14T17:18:38.123 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.994343064Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.994721706Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.997536969Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.997671321Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.997781703Z level=info msg="alerts found to migrate" alerts=0 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:37 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:37.9981956Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.000277539Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.003006379Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.003443142Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.006258177Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.00872623Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.009653688Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-14T17:18:38.124 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.010154003Z level=info msg="Executing migration" id="create data_keys table" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.011148402Z level=info msg="Executing migration" id="create secrets table" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.011857999Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.016176491Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.017119822Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.017971589Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.022342511Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.026414964Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.027261058Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.028071146Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.028878549Z level=info msg="Executing migration" id="create permission table" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.029801611Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.030261387Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.031108051Z level=info msg="Executing migration" id="create role table" 2024-02-14T17:18:38.125 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.031942889Z level=info msg="Executing migration" id="add column display_name" 2024-02-14T17:18:38.126 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.032935942Z level=info msg="Executing migration" id="add column group_name" 2024-02-14T17:18:38.126 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.033871989Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-14T17:18:38.126 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.03485615Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-14T17:18:38.126 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.035607399Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-14T17:18:38.126 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.036473265Z level=info msg="Executing migration" id="create team role table" 2024-02-14T17:18:38.126 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.036911918Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-14T17:18:38.126 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.037892267Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-14T17:18:38.126 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.038613057Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.039603603Z level=info msg="Executing migration" id="create user role table" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.040157257Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.041041311Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.041853489Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.042710899Z level=info msg="Executing migration" id="create builtin role table" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.043117025Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.044096404Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.044908611Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.04586164Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-14T17:18:38.127 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.046338197Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.047157228Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.050046229Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.052805651Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.053223727Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.056047766Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.058974993Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.061763612Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.0622516Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.065204135Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-14T17:18:38.128 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.068057213Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.070808042Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.07120255Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.074035298Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.076944769Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.080121679Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.080632455Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.083370012Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.086232485Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.088790894Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.089266824Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.092145258Z level=info msg="Executing migration" id="add correlation config column" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.094928429Z level=info msg="Executing migration" id="create entity_events table" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.097483077Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-14T17:18:38.129 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.097911019Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.09800214Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.098287266Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.098374014Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.098656047Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.101402931Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.101858582Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.102334939Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.105159535Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.107827574Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.110400685Z level=info msg="Executing migration" id="Drop public config table" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.110868205Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.111683691Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.11221975Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-14T17:18:38.130 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.112721105Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-14T17:18:38.131 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.11357041Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-14T17:18:38.172 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.y.service' 2024-02-14T17:18:38.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:18:38.208 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-14T17:18:38.208 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-14T17:18:38.208 DEBUG:teuthology.orchestra.run.smithi091:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.x 2024-02-14T17:18:38.537 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:18:38 smithi091 systemd[1]: Stopping Ceph mgr.x for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.116671744Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.117704556Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.11874802Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.1196227Z level=info msg="Executing migration" id="add share column" 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.120620096Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.121488406Z level=info msg="Executing migration" id="create file table" 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.122465781Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.123160505Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.123666586Z level=info msg="Executing migration" id="create file_meta table" 2024-02-14T17:18:38.538 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.124075012Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.124916062Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.125860089Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.126377277Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.127259347Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.128128061Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.129201829Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.129538216Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.13048156Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.13130476Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.132162592Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.132602408Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.133644321Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-14T17:18:38.539 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.134729665Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.135596428Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.145299077Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.14613436Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.146984931Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.150630305Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.151146785Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.152021343Z level=info msg="Executing migration" id="create folder table" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.152997944Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.155847242Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.156394185Z level=info msg="Executing migration" id="Update folder title length" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.159225254Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=migrator t=2024-02-14T17:18:38.162206582Z level=info msg="migrations completed" performed=485 skipped=0 duration=652.179704ms 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=sqlstore t=2024-02-14T17:18:38.162787122Z level=info msg="Created default organization" 2024-02-14T17:18:38.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=plugin.loader t=2024-02-14T17:18:38.217794978Z level=info msg="Plugin registered" pluginID=input 2024-02-14T17:18:38.541 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=plugin.loader t=2024-02-14T17:18:38.24622499Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-14T17:18:38.541 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=plugin.loader t=2024-02-14T17:18:38.246408635Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-14T17:18:38.541 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=secrets t=2024-02-14T17:18:38.246453897Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-14T17:18:38.541 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=query_data t=2024-02-14T17:18:38.248680202Z level=info msg="Query Service initialization" 2024-02-14T17:18:38.541 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=live.push_http t=2024-02-14T17:18:38.252210291Z level=info msg="Live Push Gateway initialization" 2024-02-14T17:18:38.813 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:18:38 smithi091 podman[43048]: 2024-02-14 17:18:38.536653523 +0000 UTC m=+0.122976755 container died e217a9aaa51d90e1aa11d933aee1446de61d9c3fefe2c0762dd7aeed037e404e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:18:39.103 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:18:38 smithi091 podman[43048]: 2024-02-14 17:18:38.886439289 +0000 UTC m=+0.472762520 container cleanup e217a9aaa51d90e1aa11d933aee1446de61d9c3fefe2c0762dd7aeed037e404e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5) 2024-02-14T17:18:39.104 INFO:journalctl@ceph.mgr.x.smithi091.stdout:Feb 14 17:18:38 smithi091 bash[43048]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-x 2024-02-14T17:18:39.104 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=infra.usagestats.collector t=2024-02-14T17:18:38.899653049Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-14T17:18:39.104 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=provisioning.datasources t=2024-02-14T17:18:38.900686792Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-14T17:18:39.104 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=provisioning.datasources t=2024-02-14T17:18:38.911940168Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-14T17:18:39.104 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=provisioning.alerting t=2024-02-14T17:18:38.923474945Z level=info msg="starting to provision alerting" 2024-02-14T17:18:39.104 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=provisioning.alerting t=2024-02-14T17:18:38.923519135Z level=info msg="finished to provision alerting" 2024-02-14T17:18:39.104 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=grafanaStorageLogger t=2024-02-14T17:18:38.923662132Z level=info msg="storage starting" 2024-02-14T17:18:39.104 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=ngalert.state.manager t=2024-02-14T17:18:38.92394818Z level=info msg="Warming state cache for startup" 2024-02-14T17:18:39.104 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=ngalert.state.manager t=2024-02-14T17:18:38.924109324Z level=info msg="State cache has been initialized" states=0 duration=160.547µs 2024-02-14T17:18:39.105 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=ticker t=2024-02-14T17:18:38.924214809Z level=info msg=starting first_tick=2024-02-14T17:18:40Z 2024-02-14T17:18:39.105 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=ngalert.multiorg.alertmanager t=2024-02-14T17:18:38.924219194Z level=info msg="starting MultiOrg Alertmanager" 2024-02-14T17:18:39.105 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=http.server t=2024-02-14T17:18:38.925808186Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-14T17:18:39.105 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=sqlstore.transactions t=2024-02-14T17:18:38.934898766Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-14T17:18:39.105 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=sqlstore.transactions t=2024-02-14T17:18:38.940584829Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-14T17:18:39.105 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:18:38 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a[42962]: logger=sqlstore.transactions t=2024-02-14T17:18:38.952145592Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-02-14T17:18:39.367 DEBUG:teuthology.orchestra.run.smithi091:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.x.service' 2024-02-14T17:18:39.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:18:39.405 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-14T17:18:39.405 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-14T17:18:39.405 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.z 2024-02-14T17:18:39.773 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:18:39 smithi120 systemd[1]: Stopping Ceph mgr.z for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:18:40.032 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:18:39 smithi120 podman[44187]: 2024-02-14 17:18:39.771731301 +0000 UTC m=+0.181490716 container died c9754de8f5c2029a8e120e3f03b625ae4eb5f09ffebb6ab84531ae833896f5ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-14T17:18:40.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:18:40 smithi120 podman[44187]: 2024-02-14 17:18:40.122944475 +0000 UTC m=+0.532703929 container cleanup c9754de8f5c2029a8e120e3f03b625ae4eb5f09ffebb6ab84531ae833896f5ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-14T17:18:40.305 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Feb 14 17:18:40 smithi120 bash[44187]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-mgr-z 2024-02-14T17:18:40.545 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@mgr.z.service' 2024-02-14T17:18:40.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:18:40.581 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-14T17:18:40.581 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-14T17:18:40.581 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.0 2024-02-14T17:18:40.878 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:40 smithi082 systemd[1]: Stopping Ceph osd.0 for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:18:41.232 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:40 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0[44024]: 2024-02-14T17:18:40.876+0000 7fc87f2cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:18:41.232 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:40 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0[44024]: 2024-02-14T17:18:40.876+0000 7fc87f2cd700 -1 osd.0 31 *** Got signal Terminated *** 2024-02-14T17:18:41.232 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:40 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0[44024]: 2024-02-14T17:18:40.876+0000 7fc87f2cd700 -1 osd.0 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T17:18:46.543 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:46 smithi082 podman[60031]: 2024-02-14 17:18:46.265788497 +0000 UTC m=+5.500918322 container died 9aab069714768bf0bbe2ccf14476f7059a87ccfe639f563032dec0d431f47b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS) 2024-02-14T17:18:46.892 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:46 smithi082 podman[60031]: 2024-02-14 17:18:46.698800843 +0000 UTC m=+5.933930716 container cleanup 9aab069714768bf0bbe2ccf14476f7059a87ccfe639f563032dec0d431f47b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0) 2024-02-14T17:18:46.892 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:46 smithi082 bash[60031]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0 2024-02-14T17:18:47.231 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:46 smithi082 podman[60065]: 2024-02-14 17:18:46.891819586 +0000 UTC m=+0.625029697 container remove 9aab069714768bf0bbe2ccf14476f7059a87ccfe639f563032dec0d431f47b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T17:18:47.579 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:47 smithi082 podman[60162]: 2024-02-14 17:18:47.218822153 +0000 UTC m=+0.018677009 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:18:47.579 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:47 smithi082 podman[60162]: 2024-02-14 17:18:47.342632485 +0000 UTC m=+0.142487337 container create 3918a6dabe6ec749c7bca4931394bf81586b81cdf79b6d777b9574e29a816684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-14T17:18:47.579 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:47 smithi082 podman[60162]: 2024-02-14 17:18:47.578304828 +0000 UTC m=+0.378159680 container init 3918a6dabe6ec749c7bca4931394bf81586b81cdf79b6d777b9574e29a816684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:18:47.875 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:47 smithi082 podman[60162]: 2024-02-14 17:18:47.584320192 +0000 UTC m=+0.384175044 container start 3918a6dabe6ec749c7bca4931394bf81586b81cdf79b6d777b9574e29a816684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-14T17:18:47.875 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:47 smithi082 podman[60162]: 2024-02-14 17:18:47.669818523 +0000 UTC m=+0.469673387 container attach 3918a6dabe6ec749c7bca4931394bf81586b81cdf79b6d777b9574e29a816684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True) 2024-02-14T17:18:48.173 INFO:journalctl@ceph.osd.0.smithi082.stdout:Feb 14 17:18:47 smithi082 podman[60162]: 2024-02-14 17:18:47.875093364 +0000 UTC m=+0.674948221 container died 3918a6dabe6ec749c7bca4931394bf81586b81cdf79b6d777b9574e29a816684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-14T17:18:48.550 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.0.service' 2024-02-14T17:18:48.591 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:18:48.592 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-14T17:18:48.592 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-14T17:18:48.592 DEBUG:teuthology.orchestra.run.smithi091:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.1 2024-02-14T17:18:48.895 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:48 smithi091 systemd[1]: Stopping Ceph osd.1 for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:18:50.603 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:50 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1[34858]: 2024-02-14T17:18:50.315+0000 7fdcf6bd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:18:50.604 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:50 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1[34858]: 2024-02-14T17:18:50.315+0000 7fdcf6bd0700 -1 osd.1 31 *** Got signal Terminated *** 2024-02-14T17:18:50.604 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:50 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1[34858]: 2024-02-14T17:18:50.315+0000 7fdcf6bd0700 -1 osd.1 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T17:18:56.023 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:55 smithi091 podman[43192]: 2024-02-14 17:18:55.735037375 +0000 UTC m=+6.939499522 container died 61f011004186eea474f9ec68b3af396b7ae835e4839d9d3a4457cc9b98f8985f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-14T17:18:56.353 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:56 smithi091 podman[43192]: 2024-02-14 17:18:56.096441849 +0000 UTC m=+7.300903987 container cleanup 61f011004186eea474f9ec68b3af396b7ae835e4839d9d3a4457cc9b98f8985f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-14T17:18:56.353 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:56 smithi091 bash[43192]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1 2024-02-14T17:18:56.690 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:56 smithi091 podman[43216]: 2024-02-14 17:18:56.364123131 +0000 UTC m=+0.623822012 container remove 61f011004186eea474f9ec68b3af396b7ae835e4839d9d3a4457cc9b98f8985f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-14T17:18:56.975 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:56 smithi091 podman[43310]: 2024-02-14 17:18:56.689805238 +0000 UTC m=+0.100914823 container create 6f43c00e29512529f57157d54d3dc617748c9c53c295b28644b1e347c10bc6b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-02-14T17:18:56.976 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:56 smithi091 podman[43310]: 2024-02-14 17:18:56.606378734 +0000 UTC m=+0.017488326 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:18:56.976 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:56 smithi091 podman[43310]: 2024-02-14 17:18:56.892084391 +0000 UTC m=+0.303193988 container init 6f43c00e29512529f57157d54d3dc617748c9c53c295b28644b1e347c10bc6b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-02-14T17:18:56.976 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:56 smithi091 podman[43310]: 2024-02-14 17:18:56.897638903 +0000 UTC m=+0.308748495 container start 6f43c00e29512529f57157d54d3dc617748c9c53c295b28644b1e347c10bc6b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:18:56.976 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:56 smithi091 podman[43310]: 2024-02-14 17:18:56.975155768 +0000 UTC m=+0.386265369 container attach 6f43c00e29512529f57157d54d3dc617748c9c53c295b28644b1e347c10bc6b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:18:57.353 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:57 smithi091 podman[43310]: 2024-02-14 17:18:57.195534654 +0000 UTC m=+0.606644240 container died 6f43c00e29512529f57157d54d3dc617748c9c53c295b28644b1e347c10bc6b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-14T17:18:57.687 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:57 smithi091 podman[43355]: 2024-02-14 17:18:57.669388133 +0000 UTC m=+0.468691335 container remove 6f43c00e29512529f57157d54d3dc617748c9c53c295b28644b1e347c10bc6b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-14T17:18:57.687 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:57 smithi091 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.1.service: Deactivated successfully. 2024-02-14T17:18:57.687 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:57 smithi091 systemd[1]: Stopped Ceph osd.1 for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:18:57.687 INFO:journalctl@ceph.osd.1.smithi091.stdout:Feb 14 17:18:57 smithi091 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.1.service: Consumed 3.954s CPU time. 2024-02-14T17:18:57.689 DEBUG:teuthology.orchestra.run.smithi091:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.1.service' 2024-02-14T17:18:57.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:18:57.729 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-14T17:18:57.729 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-14T17:18:57.729 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.2 2024-02-14T17:18:58.029 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:18:57 smithi120 systemd[1]: Stopping Ceph osd.2 for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:18:58.305 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:18:58 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2[36350]: 2024-02-14T17:18:58.027+0000 7efea5210700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:18:58.305 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:18:58 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2[36350]: 2024-02-14T17:18:58.027+0000 7efea5210700 -1 osd.2 31 *** Got signal Terminated *** 2024-02-14T17:18:58.305 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:18:58 smithi120 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2[36350]: 2024-02-14T17:18:58.027+0000 7efea5210700 -1 osd.2 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T17:19:03.683 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:03 smithi120 podman[44330]: 2024-02-14 17:19:03.381549147 +0000 UTC m=+5.458599498 container died e9f6b24ab5aa9b1ed3cd3091d0f32229760efa55a1587d8bf1ee812c3c717913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T17:19:03.683 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:03 smithi120 podman[44330]: 2024-02-14 17:19:03.682475921 +0000 UTC m=+5.759526277 container cleanup e9f6b24ab5aa9b1ed3cd3091d0f32229760efa55a1587d8bf1ee812c3c717913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-14T17:19:04.055 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:03 smithi120 bash[44330]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2 2024-02-14T17:19:04.055 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:03 smithi120 podman[44352]: 2024-02-14 17:19:03.900001769 +0000 UTC m=+0.513803950 container remove e9f6b24ab5aa9b1ed3cd3091d0f32229760efa55a1587d8bf1ee812c3c717913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-14T17:19:04.486 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:04 smithi120 podman[44449]: 2024-02-14 17:19:04.217456917 +0000 UTC m=+0.109999193 container create 6f2e59b6cec63630cd24e254517aed196f32ece77da15a81e315aafd0f419bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-14T17:19:04.487 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:04 smithi120 podman[44449]: 2024-02-14 17:19:04.12512826 +0000 UTC m=+0.017670498 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:19:04.487 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:04 smithi120 podman[44449]: 2024-02-14 17:19:04.394472806 +0000 UTC m=+0.287015035 container init 6f2e59b6cec63630cd24e254517aed196f32ece77da15a81e315aafd0f419bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T17:19:04.487 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:04 smithi120 podman[44449]: 2024-02-14 17:19:04.400169453 +0000 UTC m=+0.292711687 container start 6f2e59b6cec63630cd24e254517aed196f32ece77da15a81e315aafd0f419bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:19:04.487 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:04 smithi120 podman[44449]: 2024-02-14 17:19:04.485905672 +0000 UTC m=+0.378447923 container attach 6f2e59b6cec63630cd24e254517aed196f32ece77da15a81e315aafd0f419bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-14T17:19:04.805 INFO:journalctl@ceph.osd.2.smithi120.stdout:Feb 14 17:19:04 smithi120 podman[44449]: 2024-02-14 17:19:04.689983049 +0000 UTC m=+0.582525294 container died 6f2e59b6cec63630cd24e254517aed196f32ece77da15a81e315aafd0f419bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-14T17:19:05.352 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@osd.2.service' 2024-02-14T17:19:05.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:19:05.391 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-14T17:19:05.392 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-14T17:19:05.392 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@rgw.foo.a 2024-02-14T17:19:05.693 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:19:05 smithi082 systemd[1]: Stopping Ceph rgw.foo.a for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:19:05.982 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:19:05 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a[50576]: 2024-02-14T17:19:05.691+0000 7fe1ea352700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:19:05.982 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:19:05 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a[50576]: 2024-02-14T17:19:05.691+0000 7fe1f8bc3ac0 -1 shutting down 2024-02-14T17:19:15.983 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:19:15 smithi082 bash[60293]: time="2024-02-14T17:19:15Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-14T17:19:15.984 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:19:15 smithi082 podman[60293]: 2024-02-14 17:19:15.712649445 +0000 UTC m=+10.126390510 container died 52f0cd3e4f0ddfebeacb0b3efb8dfd71d409001890571fdf472289194266949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:19:16.331 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:19:16 smithi082 podman[60293]: 2024-02-14 17:19:16.153933336 +0000 UTC m=+10.567674402 container cleanup 52f0cd3e4f0ddfebeacb0b3efb8dfd71d409001890571fdf472289194266949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-14T17:19:16.331 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:19:16 smithi082 bash[60293]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a 2024-02-14T17:19:16.583 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:19:16 smithi082 podman[60318]: 2024-02-14 17:19:16.330201859 +0000 UTC m=+0.611930450 container remove 52f0cd3e4f0ddfebeacb0b3efb8dfd71d409001890571fdf472289194266949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-rgw-foo-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-02-14T17:19:16.584 INFO:journalctl@ceph.rgw.foo.a.smithi082.stdout:Feb 14 17:19:16 smithi082 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-02-14T17:19:16.587 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@rgw.foo.a.service' 2024-02-14T17:19:16.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:19:16.626 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-14T17:19:16.626 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-14T17:19:16.626 DEBUG:teuthology.orchestra.run.smithi091:> sudo systemctl stop ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@prometheus.a 2024-02-14T17:19:16.922 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 systemd[1]: Stopping Ceph prometheus.a for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:19:17.211 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a[40915]: ts=2024-02-14T17:19:16.921Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-14T17:19:17.212 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:16 smithi091 podman[43440]: 2024-02-14 17:19:16.931495872 +0000 UTC m=+0.116251465 container died 829071073b8136ad97c67af30409e518c8d429a3277d9a59c873a13b5ef9b851 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:19:17.553 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:17 smithi091 podman[43440]: 2024-02-14 17:19:17.335316545 +0000 UTC m=+0.520072140 container cleanup 829071073b8136ad97c67af30409e518c8d429a3277d9a59c873a13b5ef9b851 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:19:17.553 INFO:journalctl@ceph.prometheus.a.smithi091.stdout:Feb 14 17:19:17 smithi091 bash[43440]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-prometheus-a 2024-02-14T17:19:17.766 DEBUG:teuthology.orchestra.run.smithi091:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@prometheus.a.service' 2024-02-14T17:19:17.801 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T17:19:17.801 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-14T17:19:17.801 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T17:19:17.992 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:19:18.981 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:19:18 smithi082 systemd[1]: Stopping Ceph alertmanager.a for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:19:19.368 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:19:19 smithi082 ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a[58196]: ts=2024-02-14T17:19:19.086Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-14T17:19:19.368 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:19:19 smithi082 podman[60648]: 2024-02-14 17:19:19.10686364 +0000 UTC m=+0.129962840 container died 003d103538210d4df03a77d1ebe013fc04670ae08966a9c45378908bea5f9b09 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:19:19.702 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:19:19 smithi082 podman[60648]: 2024-02-14 17:19:19.541673372 +0000 UTC m=+0.564772570 container cleanup 003d103538210d4df03a77d1ebe013fc04670ae08966a9c45378908bea5f9b09 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:19:19.702 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:19:19 smithi082 bash[60648]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a 2024-02-14T17:19:19.981 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:19:19 smithi082 podman[60667]: 2024-02-14 17:19:19.701282087 +0000 UTC m=+0.597357706 container remove 003d103538210d4df03a77d1ebe013fc04670ae08966a9c45378908bea5f9b09 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:19:19.981 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:19:19 smithi082 podman[60667]: 2024-02-14 17:19:19.785028921 +0000 UTC m=+0.681104492 volume remove 0c3ca61a108d8f9ce81a12deb4a8e8a02ce77c96f901081b1cb4d439867832f5 2024-02-14T17:19:20.288 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:19:20 smithi082 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-14T17:19:20.289 INFO:journalctl@ceph.alertmanager.a.smithi082.stdout:Feb 14 17:19:20 smithi082 systemd[1]: Stopped Ceph alertmanager.a for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:19:20.640 DEBUG:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T17:19:20.823 INFO:teuthology.orchestra.run.smithi091.stdout:Deleting cluster with fsid: 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:19:21.928 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:19:21 smithi091 systemd[1]: Stopping Ceph grafana.a for 39c4996e-cb5c-11ee-95b9-87774f69a715... 2024-02-14T17:19:22.206 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:19:21 smithi091 podman[43789]: 2024-02-14 17:19:21.927509794 +0000 UTC m=+0.119200503 container died 403d55489ae7a955cf6bf062151dba7ffccdbaa79a8b2ba017c014c2bd5e8be6 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , distribution-scope=public, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, io.openshift.tags=base rhel8, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.32.0, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., architecture=x86_64, description=Ceph Grafana Container, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container) 2024-02-14T17:19:22.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:19:22 smithi091 podman[43789]: 2024-02-14 17:19:22.279970484 +0000 UTC m=+0.471661220 container cleanup 403d55489ae7a955cf6bf062151dba7ffccdbaa79a8b2ba017c014c2bd5e8be6 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., io.buildah.version=1.32.0, architecture=x86_64, maintainer=Paul Cuzner , com.redhat.component=ubi8-container, io.openshift.expose-services=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, distribution-scope=public, vcs-type=git, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-02-14T17:19:22.540 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:19:22 smithi091 bash[43789]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a 2024-02-14T17:19:22.844 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:19:22 smithi091 podman[43808]: 2024-02-14 17:19:22.539385526 +0000 UTC m=+0.614761947 container remove 403d55489ae7a955cf6bf062151dba7ffccdbaa79a8b2ba017c014c2bd5e8be6 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-39c4996e-cb5c-11ee-95b9-87774f69a715-grafana-a, description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, release=239.1651231664, name=ubi8, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, distribution-scope=public, maintainer=Paul Cuzner , vcs-type=git, io.openshift.expose-services=, vendor=Red Hat, Inc., version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-02-14T17:19:22.844 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:19:22 smithi091 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-14T17:19:22.844 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:19:22 smithi091 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-14T17:19:22.844 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:19:22 smithi091 systemd[1]: Stopped Ceph grafana.a for 39c4996e-cb5c-11ee-95b9-87774f69a715. 2024-02-14T17:19:22.844 INFO:journalctl@ceph.grafana.a.smithi091.stdout:Feb 14 17:19:22 smithi091 systemd[1]: ceph-39c4996e-cb5c-11ee-95b9-87774f69a715@grafana.a.service: Consumed 2.632s CPU time. 2024-02-14T17:19:23.395 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T17:19:23.584 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:19:24.673 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:19:24.708 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:19:24.743 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:19:24.772 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-14T17:19:24.774 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/remote/smithi082/crash 2024-02-14T17:19:24.774 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/crash -- . 2024-02-14T17:19:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T17:19:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2024-02-14T17:19:24.823 DEBUG:teuthology.misc:Transferring archived files from smithi091:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/remote/smithi091/crash 2024-02-14T17:19:24.824 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar c -f - -C /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/crash -- . 2024-02-14T17:19:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:tar: /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T17:19:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:tar: Error is not recoverable: exiting now 2024-02-14T17:19:24.874 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/remote/smithi120/crash 2024-02-14T17:19:24.875 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/crash -- . 2024-02-14T17:19:24.913 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T17:19:24.913 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now 2024-02-14T17:19:24.914 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-14T17:19:24.914 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-14T17:19:24.946 INFO:teuthology.orchestra.run.smithi082.stdout:2024-02-14T17:15:31.728729+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-14T17:19:24.946 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-14T17:19:24.947 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[SEC\]' /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-14T17:19:25.017 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]' /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-14T17:19:25.087 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[WRN\]' /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-14T17:19:25.155 INFO:teuthology.orchestra.run.smithi082.stdout:2024-02-14T17:15:31.728729+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-14T17:19:25.156 INFO:tasks.cephadm:Compressing logs... 2024-02-14T17:19:25.156 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T17:19:25.198 DEBUG:teuthology.orchestra.run.smithi091:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T17:19:25.200 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T17:19:25.223 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-14T17:19:25.223 INFO:teuthology.orchestra.run.smithi082.stderr:: No such file or directory 2024-02-14T17:19:25.223 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mon.a.log 2024-02-14T17:19:25.223 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T17:19:25.223 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-14T17:19:25.223 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mon.a.log: /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mgr.y.log 2024-02-14T17:19:25.225 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-osd.0.log 2024-02-14T17:19:25.225 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log: 57.8% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-14T17:19:25.225 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T17:19:25.225 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log 2024-02-14T17:19:25.226 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T17:19:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T17:19:25.226 INFO:teuthology.orchestra.run.smithi082.stderr: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T17:19:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log: 82.7% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T17:19:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-volume.log: 85.0% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T17:19:25.229 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-14T17:19:25.229 INFO:teuthology.orchestra.run.smithi120.stderr:: No such file or directory 2024-02-14T17:19:25.229 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T17:19:25.229 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mgr.z.log 2024-02-14T17:19:25.229 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T17:19:25.229 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log 2024-02-14T17:19:25.229 INFO:teuthology.orchestra.run.smithi120.stderr: 90.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T17:19:25.230 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-osd.2.log 2024-02-14T17:19:25.230 INFO:teuthology.orchestra.run.smithi120.stderr: 81.2% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T17:19:25.230 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T17:19:25.230 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log: 89.4% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T17:19:25.230 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mon.c.log 2024-02-14T17:19:25.230 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-volume.log: 84.6% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T17:19:25.230 INFO:teuthology.orchestra.run.smithi082.stderr: 90.5% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T17:19:25.231 INFO:teuthology.orchestra.run.smithi091.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-02-14T17:19:25.231 INFO:teuthology.orchestra.run.smithi091.stderr: 2024-02-14T17:19:25.231 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T17:19:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-osd.1.log 2024-02-14T17:19:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mgr.x.log 2024-02-14T17:19:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T17:19:25.233 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mon.c.log: 90.1% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mgr.z.log.gz 2024-02-14T17:19:25.233 INFO:teuthology.orchestra.run.smithi120.stderr: 90.5% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T17:19:25.234 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log 2024-02-14T17:19:25.234 INFO:teuthology.orchestra.run.smithi091.stderr: 89.3% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T17:19:25.234 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T17:19:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log: 81.3% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T17:19:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mon.b.log 2024-02-14T17:19:25.235 INFO:teuthology.orchestra.run.smithi091.stderr: 81.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T17:19:25.235 INFO:teuthology.orchestra.run.smithi091.stderr: 84.5%/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-volume.log: -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T17:19:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mon.b.log: 90.2% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mgr.x.log.gz 2024-02-14T17:19:25.235 INFO:teuthology.orchestra.run.smithi091.stderr: 90.5% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T17:19:25.284 INFO:teuthology.orchestra.run.smithi091.stderr: 92.2% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mon.b.log.gz 2024-02-14T17:19:25.285 INFO:teuthology.orchestra.run.smithi120.stderr: 92.2% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mon.c.log.gz 2024-02-14T17:19:25.299 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mgr.y.log.gz 2024-02-14T17:19:25.389 INFO:teuthology.orchestra.run.smithi082.stderr: 91.3% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-mon.a.log.gz 2024-02-14T17:19:25.479 INFO:teuthology.orchestra.run.smithi082.stderr: 94.3% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-osd.0.log.gz 2024-02-14T17:19:25.481 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-02-14T17:19:25.482 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m0.271s 2024-02-14T17:19:25.482 INFO:teuthology.orchestra.run.smithi082.stderr:user 0m0.497s 2024-02-14T17:19:25.482 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m0.039s 2024-02-14T17:19:25.484 INFO:teuthology.orchestra.run.smithi120.stderr: 94.3% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-osd.2.log.gz 2024-02-14T17:19:25.486 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-02-14T17:19:25.487 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m0.271s 2024-02-14T17:19:25.487 INFO:teuthology.orchestra.run.smithi120.stderr:user 0m0.321s 2024-02-14T17:19:25.487 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m0.030s 2024-02-14T17:19:25.524 INFO:teuthology.orchestra.run.smithi091.stderr: 94.3% -- replaced with /var/log/ceph/39c4996e-cb5c-11ee-95b9-87774f69a715/ceph-osd.1.log.gz 2024-02-14T17:19:25.526 INFO:teuthology.orchestra.run.smithi091.stderr: 2024-02-14T17:19:25.526 INFO:teuthology.orchestra.run.smithi091.stderr:real 0m0.312s 2024-02-14T17:19:25.526 INFO:teuthology.orchestra.run.smithi091.stderr:user 0m0.354s 2024-02-14T17:19:25.527 INFO:teuthology.orchestra.run.smithi091.stderr:sys 0m0.039s 2024-02-14T17:19:25.527 INFO:tasks.cephadm:Archiving logs... 2024-02-14T17:19:25.527 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/remote/smithi082/log 2024-02-14T17:19:25.528 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T17:19:25.659 DEBUG:teuthology.misc:Transferring archived files from smithi091:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/remote/smithi091/log 2024-02-14T17:19:25.659 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T17:19:25.744 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/remote/smithi120/log 2024-02-14T17:19:25.745 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T17:19:25.830 INFO:tasks.cephadm:Removing cluster... 2024-02-14T17:19:25.831 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 --force 2024-02-14T17:19:26.008 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:19:26.122 DEBUG:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 --force 2024-02-14T17:19:26.302 INFO:teuthology.orchestra.run.smithi091.stdout:Deleting cluster with fsid: 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:19:26.416 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39c4996e-cb5c-11ee-95b9-87774f69a715 --force 2024-02-14T17:19:26.599 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 39c4996e-cb5c-11ee-95b9-87774f69a715 2024-02-14T17:19:26.714 INFO:tasks.cephadm:Removing cephadm ... 2024-02-14T17:19:26.714 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-14T17:19:26.729 DEBUG:teuthology.orchestra.run.smithi091:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-14T17:19:26.745 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-14T17:19:26.760 INFO:tasks.cephadm:Teardown complete 2024-02-14T17:19:26.761 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-02-14T17:19:26.772 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi082:/dev/vg_nvme/lv_1... 2024-02-14T17:19:26.772 DEBUG:teuthology.orchestra.run.smithi082:> sudo nvme disconnect -n lv_1 2024-02-14T17:19:26.915 INFO:teuthology.orchestra.run.smithi082.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-14T17:19:26.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:26.917 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi082:/dev/vg_nvme/lv_2... 2024-02-14T17:19:26.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo nvme disconnect -n lv_2 2024-02-14T17:19:27.099 INFO:teuthology.orchestra.run.smithi082.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-14T17:19:27.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:27.100 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi082:/dev/vg_nvme/lv_3... 2024-02-14T17:19:27.100 DEBUG:teuthology.orchestra.run.smithi082:> sudo nvme disconnect -n lv_3 2024-02-14T17:19:27.278 INFO:teuthology.orchestra.run.smithi082.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-14T17:19:27.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:27.280 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi082:/dev/vg_nvme/lv_4... 2024-02-14T17:19:27.280 DEBUG:teuthology.orchestra.run.smithi082:> sudo nvme disconnect -n lv_4 2024-02-14T17:19:27.473 INFO:teuthology.orchestra.run.smithi082.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-14T17:19:27.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:27.475 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-14T17:19:27.475 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/scratch_devs 2024-02-14T17:19:27.501 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi091:/dev/vg_nvme/lv_1... 2024-02-14T17:19:27.501 DEBUG:teuthology.orchestra.run.smithi091:> sudo nvme disconnect -n lv_1 2024-02-14T17:19:27.642 INFO:teuthology.orchestra.run.smithi091.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-14T17:19:27.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:27.644 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi091:/dev/vg_nvme/lv_2... 2024-02-14T17:19:27.645 DEBUG:teuthology.orchestra.run.smithi091:> sudo nvme disconnect -n lv_2 2024-02-14T17:19:27.786 INFO:teuthology.orchestra.run.smithi091.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-14T17:19:27.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:27.788 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi091:/dev/vg_nvme/lv_3... 2024-02-14T17:19:27.788 DEBUG:teuthology.orchestra.run.smithi091:> sudo nvme disconnect -n lv_3 2024-02-14T17:19:27.950 INFO:teuthology.orchestra.run.smithi091.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-14T17:19:27.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:27.952 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi091:/dev/vg_nvme/lv_4... 2024-02-14T17:19:27.953 DEBUG:teuthology.orchestra.run.smithi091:> sudo nvme disconnect -n lv_4 2024-02-14T17:19:28.089 INFO:teuthology.orchestra.run.smithi091.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-14T17:19:28.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:28.091 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-14T17:19:28.091 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/scratch_devs 2024-02-14T17:19:28.122 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi120:/dev/vg_nvme/lv_1... 2024-02-14T17:19:28.122 DEBUG:teuthology.orchestra.run.smithi120:> sudo nvme disconnect -n lv_1 2024-02-14T17:19:28.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:28.273 INFO:teuthology.orchestra.run.smithi120.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-14T17:19:28.273 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi120:/dev/vg_nvme/lv_2... 2024-02-14T17:19:28.274 DEBUG:teuthology.orchestra.run.smithi120:> sudo nvme disconnect -n lv_2 2024-02-14T17:19:28.421 INFO:teuthology.orchestra.run.smithi120.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-14T17:19:28.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:28.423 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi120:/dev/vg_nvme/lv_3... 2024-02-14T17:19:28.423 DEBUG:teuthology.orchestra.run.smithi120:> sudo nvme disconnect -n lv_3 2024-02-14T17:19:28.551 INFO:teuthology.orchestra.run.smithi120.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-14T17:19:28.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:28.553 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi120:/dev/vg_nvme/lv_4... 2024-02-14T17:19:28.553 DEBUG:teuthology.orchestra.run.smithi120:> sudo nvme disconnect -n lv_4 2024-02-14T17:19:28.700 INFO:teuthology.orchestra.run.smithi120.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-14T17:19:28.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:28.702 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-02-14T17:19:28.702 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/scratch_devs 2024-02-14T17:19:28.731 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-14T17:19:28.743 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-14T17:19:28.743 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:19:28.746 DEBUG:teuthology.orchestra.run.smithi091:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:19:28.749 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:19:28.761 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:19:28.763 INFO:teuthology.orchestra.run.smithi091.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:19:28.765 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:19:28.765 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-02-14T17:19:28.765 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +1836us[+1836us] +/- 127ms 2024-02-14T17:19:28.765 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -689us[ -679us] +/- 53ms 2024-02-14T17:19:28.765 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -78us[ -78us] +/- 81ms 2024-02-14T17:19:28.765 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:19:28.767 INFO:teuthology.orchestra.run.smithi091.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:19:28.767 INFO:teuthology.orchestra.run.smithi091.stdout:=============================================================================== 2024-02-14T17:19:28.767 INFO:teuthology.orchestra.run.smithi091.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 +1846us[+1843us] +/- 127ms 2024-02-14T17:19:28.767 INFO:teuthology.orchestra.run.smithi091.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 -681us[ -685us] +/- 53ms 2024-02-14T17:19:28.767 INFO:teuthology.orchestra.run.smithi091.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -75us[ -75us] +/- 81ms 2024-02-14T17:19:28.767 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:19:28.786 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:19:28.791 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:19:28.791 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-02-14T17:19:28.791 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 +1844us[+1844us] +/- 127ms 2024-02-14T17:19:28.791 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 -682us[ -701us] +/- 53ms 2024-02-14T17:19:28.792 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -80us[ -92us] +/- 81ms 2024-02-14T17:19:28.792 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:19:28.792 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-14T17:19:28.804 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-14T17:19:28.805 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-14T17:19:28.818 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T17:19:28.897 DEBUG:teuthology.orchestra.run.smithi091:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T17:19:28.970 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T17:19:29.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:19:29.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:29.077 DEBUG:teuthology.orchestra.run.smithi091:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:19:29.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:29.109 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:19:29.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:29.140 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-14T17:19:29.152 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-14T17:19:29.167 INFO:teuthology.task.internal:Duration was 687.312054 seconds 2024-02-14T17:19:29.167 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-14T17:19:29.186 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-14T17:19:29.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T17:19:29.190 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T17:19:29.193 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T17:19:29.233 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:19:29.234 INFO:teuthology.orchestra.run.smithi091.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:19:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:19:29.640 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-14T17:19:29.640 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-02-14T17:19:29.640 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T17:19:29.661 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi091.front.sepia.ceph.com 2024-02-14T17:19:29.662 DEBUG:teuthology.orchestra.run.smithi091:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T17:19:29.709 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-02-14T17:19:29.709 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T17:19:29.749 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-14T17:19:29.750 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T17:19:29.753 DEBUG:teuthology.orchestra.run.smithi091:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T17:19:29.756 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T17:19:29.820 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-14T17:19:29.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T17:19:29.826 DEBUG:teuthology.orchestra.run.smithi091:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T17:19:29.830 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T17:19:30.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-14T17:19:30.029 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-14T17:19:30.029 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T17:19:30.056 DEBUG:teuthology.orchestra.run.smithi091:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T17:19:30.083 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T17:19:30.110 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-14T17:19:30.126 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T17:19:30.129 DEBUG:teuthology.orchestra.run.smithi091:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T17:19:30.131 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T17:19:30.151 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-02-14T17:19:30.154 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern = core 2024-02-14T17:19:30.176 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-02-14T17:19:30.203 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T17:19:30.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:30.232 DEBUG:teuthology.orchestra.run.smithi091:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T17:19:30.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:30.246 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T17:19:30.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:19:30.260 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-14T17:19:30.270 INFO:teuthology.task.internal:Transferring archived files... 2024-02-14T17:19:30.271 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/remote/smithi082 2024-02-14T17:19:30.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T17:19:30.312 DEBUG:teuthology.misc:Transferring archived files from smithi091:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/remote/smithi091 2024-02-14T17:19:30.312 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T17:19:30.350 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760/remote/smithi120 2024-02-14T17:19:30.350 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T17:19:30.396 INFO:teuthology.task.internal:Removing archive directory... 2024-02-14T17:19:30.397 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T17:19:30.400 DEBUG:teuthology.orchestra.run.smithi091:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T17:19:30.405 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T17:19:30.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-14T17:19:30.527 INFO:teuthology.task.internal:Not uploading archives. 2024-02-14T17:19:30.528 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-14T17:19:30.611 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-14T17:19:30.611 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T17:19:30.615 DEBUG:teuthology.orchestra.run.smithi091:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T17:19:30.618 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T17:19:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 17:19 /home/ubuntu/cephtest 2024-02-14T17:19:30.631 INFO:teuthology.orchestra.run.smithi091.stdout: 265787 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 17:19 /home/ubuntu/cephtest 2024-02-14T17:19:30.633 INFO:teuthology.orchestra.run.smithi120.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 17:19 /home/ubuntu/cephtest 2024-02-14T17:19:30.634 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-14T17:19:30.678 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-14T17:19:30.713 INFO:teuthology.nuke:Checking targets against current locks 2024-02-14T17:19:30.738 DEBUG:teuthology.nuke:shortname: smithi082 2024-02-14T17:19:30.738 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T17:19:30.762 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 16:59:15.164873', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKh/VHUWQOqI6QPvCO7+14T/bSfcmUqXvrmmQp+6+fr7zVu8bTlQ+0o/m13/Fi4TFxD51bUN7n56Dvxj+2u4ht0='} 2024-02-14T17:19:30.766 DEBUG:teuthology.nuke:shortname: smithi091 2024-02-14T17:19:30.766 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T17:19:30.788 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi091.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 16:59:15.166664', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBk4YGW62+4YkJfAbNOEeGrLY1Z5VeUfQHlDS5XTU1sJLPwgxW/+R3eGh9gGUKgQckk+VIjqrwZC5w3KPfz65Lk='} 2024-02-14T17:19:30.793 DEBUG:teuthology.nuke:shortname: smithi120 2024-02-14T17:19:30.793 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T17:19:30.817 INFO:teuthology.orchestra.console.smithi082:Power off 2024-02-14T17:19:30.818 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T17:19:30.830 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559760', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 16:59:15.162886', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAl3UarpUpAr+yoD5/2J0GNylfCVpPOCn3D/R8yGbU/+ukyW9QLXTurAWJnZx1CbygADGYd/mVDvCEvxTurqu20='} 2024-02-14T17:19:30.841 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-02-14T17:19:30.841 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:30.861 INFO:teuthology.orchestra.console.smithi091:Power off 2024-02-14T17:19:30.861 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T17:19:30.872 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-02-14T17:19:30.884 DEBUG:teuthology.orchestra.console.smithi091:power off output: Chassis Power Control: Down/Off 2024-02-14T17:19:30.885 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:30.899 INFO:teuthology.orchestra.console.smithi120:Power off 2024-02-14T17:19:30.899 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T17:19:30.908 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on 2024-02-14T17:19:30.925 DEBUG:teuthology.orchestra.console.smithi120:power off output: Chassis Power Control: Down/Off 2024-02-14T17:19:30.925 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:30.954 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is on 2024-02-14T17:19:34.873 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:34.909 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:34.955 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:34.983 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-02-14T17:19:35.020 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on 2024-02-14T17:19:35.066 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is on 2024-02-14T17:19:38.989 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:39.020 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:39.066 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:39.099 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-02-14T17:19:39.130 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off 2024-02-14T17:19:39.176 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is on 2024-02-14T17:19:39.200 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-02-14T17:19:39.231 INFO:teuthology.orchestra.console.smithi091:Power off completed 2024-02-14T17:19:43.180 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T17:19:43.290 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is off 2024-02-14T17:19:43.390 INFO:teuthology.orchestra.console.smithi120:Power off completed 2024-02-14T17:19:43.495 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} duration: 687.3120539188385 failure_reason: '"2024-02-14T17:15:31.728729+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' owner: scheduled_gabrioux@teuthology success: false 2024-02-14T17:19:43.495 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T17:19:43.573 INFO:teuthology.run:FAIL