2024-02-07T13:21:46.229 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-07T13:21:46.230 DEBUG:teuthology.run:Teuthology command: teuthology --description orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} --owner scheduled_gabrioux@teuthology --name gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi --archive /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802 --verbose -- /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/orig.config.yaml 2024-02-07T13:21:46.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-07T13:21:46.345 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802 branch: wip-guits-testing-4-2024-02-05-1854 description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} email: gabrioux@ibm.com first_in_suite: false job_id: '7548802' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-4-2024-02-05-1854 ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-4-2024-02-05-1854 sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 owner: scheduled_gabrioux@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 1701 sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-4-2024-02-05-1854 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_44458ccedb18241cc2e17f10cdab3d1385399261/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOPf41Y98XLj3FwHreyOAexv4hXn/EvTZsS0oWIJ9YL92vybX289iPq9bLdloH0rpnzDapH6/ieA0Hhn1/IoN1g= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEhqXTkUlRo6RvIzmZkkeo1HcOpjPWPnEK9mI4G6XkuRdHTvqUb1J5PIkgNhgSgkHPkBaawlgr+uVZfuk6jDcFc= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFfJfD06HpTnT9dw9pbSErpDqkWD+hJYPnsZ/Nzpnjqj7C75iZPiuaql6PyO06rMwF3H1wAhImKbR7ujc00Jdrc= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-06_08:19:47 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-07T13:21:46.345 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_44458ccedb18241cc2e17f10cdab3d1385399261/qa; will attempt to use it 2024-02-07T13:21:46.346 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_44458ccedb18241cc2e17f10cdab3d1385399261/qa/tasks 2024-02-07T13:21:46.346 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-07T13:21:46.348 INFO:teuthology.task.internal:Checking packages... 2024-02-07T13:21:46.373 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '44458ccedb18241cc2e17f10cdab3d1385399261' 2024-02-07T13:21:46.373 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-07T13:21:46.373 INFO:teuthology.packaging:ref: None 2024-02-07T13:21:46.373 INFO:teuthology.packaging:tag: None 2024-02-07T13:21:46.373 INFO:teuthology.packaging:branch: wip-guits-testing-4-2024-02-05-1854 2024-02-07T13:21:46.373 INFO:teuthology.packaging:sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:21:46.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-4-2024-02-05-1854 2024-02-07T13:21:46.514 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1252.g44458cce 2024-02-07T13:21:46.516 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-07T13:21:46.521 INFO:teuthology.task.internal:no buildpackages task found 2024-02-07T13:21:46.522 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-07T13:21:46.547 INFO:teuthology.task.internal:Saving configuration 2024-02-07T13:21:46.559 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-07T13:21:46.580 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T13:21:46.605 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 13:16:37.680348', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOPf41Y98XLj3FwHreyOAexv4hXn/EvTZsS0oWIJ9YL92vybX289iPq9bLdloH0rpnzDapH6/ieA0Hhn1/IoN1g='} 2024-02-07T13:21:46.626 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 13:16:37.684149', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEhqXTkUlRo6RvIzmZkkeo1HcOpjPWPnEK9mI4G6XkuRdHTvqUb1J5PIkgNhgSgkHPkBaawlgr+uVZfuk6jDcFc='} 2024-02-07T13:21:46.653 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 13:16:37.682431', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFfJfD06HpTnT9dw9pbSErpDqkWD+hJYPnsZ/Nzpnjqj7C75iZPiuaql6PyO06rMwF3H1wAhImKbR7ujc00Jdrc='} 2024-02-07T13:21:46.653 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-07T13:21:46.659 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-07T13:21:46.660 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-02-07T13:21:46.660 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-02-07T13:21:46.660 INFO:teuthology.run_tasks:Running task console_log... 2024-02-07T13:21:46.780 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fab92d58f70>, signals=[15]) 2024-02-07T13:21:46.781 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-07T13:21:46.786 INFO:teuthology.task.internal:Opening connections... 2024-02-07T13:21:46.786 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-02-07T13:21:46.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:21:46.878 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-02-07T13:21:46.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:21:46.967 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-02-07T13:21:46.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:21:47.055 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-07T13:21:47.062 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-02-07T13:21:47.079 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-02-07T13:21:47.079 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-02-07T13:21:47.135 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-07T13:21:47.136 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-07T13:21:47.136 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-07T13:21:47.136 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-02-07T13:21:47.157 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-02-07T13:21:47.175 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-02-07T13:21:47.175 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-02-07T13:21:47.231 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-02-07T13:21:47.231 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-07T13:21:47.232 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-07T13:21:47.233 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-02-07T13:21:47.257 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-02-07T13:21:47.274 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-02-07T13:21:47.274 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-02-07T13:21:47.334 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-02-07T13:21:47.334 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-07T13:21:47.335 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-07T13:21:47.336 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-02-07T13:21:47.357 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-07T13:21:47.367 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-07T13:21:47.375 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-07T13:21:47.375 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-02-07T13:21:47.377 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-02-07T13:21:47.380 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-02-07T13:21:47.395 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-07T13:21:47.400 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-07T13:21:47.400 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-02-07T13:21:47.434 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-02-07T13:21:47.436 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-02-07T13:21:47.460 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-07T13:21:47.524 INFO:teuthology.run_tasks:Running task kernel... 2024-02-07T13:21:47.538 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-07T13:21:47.538 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-07T13:21:47.538 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-07T13:21:47.539 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-07T13:21:47.539 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-07T13:21:47.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:21:47.555 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-02-07T13:21:47.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:21:47.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:21:47.557 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-02-07T13:21:47.557 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-02-07T13:21:47.613 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-361.el9.x86_64 2024-02-07T13:21:47.614 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.14.0-361.el9.x86_64 2024-02-07T13:21:47.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-02-07T13:21:47.615 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-361.el9.x86_64 2024-02-07T13:21:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-361.el9.x86_64 2024-02-07T13:21:47.616 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.14.0-361.el9.x86_64 2024-02-07T13:21:47.616 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-02-07T13:21:47.616 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.14.0-361.el9.x86_64 2024-02-07T13:21:47.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum install -y kernel 2024-02-07T13:21:49.733 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-02-07T13:21:49.828 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-02-07T13:21:49.994 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-02-07T13:21:51.125 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-02-07T13:21:51.237 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-02-07T13:21:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-02-07T13:21:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-02-07T13:21:53.258 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.0 MB 00:01 2024-02-07T13:21:54.277 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.0 MB 00:02 2024-02-07T13:21:55.057 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-02-07T13:21:55.292 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-02-07T13:21:57.172 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-02-07T13:21:59.172 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 5.9 MB 00:00 2024-02-07T13:21:59.543 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 5.9 MB 00:00 2024-02-07T13:22:00.756 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2024-02-07T13:22:00.868 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 15 kB 00:00 2024-02-07T13:22:01.053 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 13 MB/s | 5.9 MB 00:00 2024-02-07T13:22:01.742 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2024-02-07T13:22:02.471 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 15 kB 00:00 2024-02-07T13:22:02.730 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-07T13:22:04.847 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 20 MB 00:02 2024-02-07T13:22:06.625 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-02-07T13:22:07.561 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-02-07T13:22:08.843 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-07T13:22:08.843 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-07T13:22:08.945 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-02-07T13:22:08.956 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-02-07T13:22:08.956 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-02-07T13:22:08.956 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-02-07T13:22:08.956 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-02-07T13:22:08.956 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-418.el9 CentOS-BaseOS 6.0 M 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core x86_64 5.14.0-418.el9 CentOS-BaseOS 21 M 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules x86_64 5.14.0-418.el9 CentOS-BaseOS 39 M 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core x86_64 5.14.0-418.el9 CentOS-BaseOS 33 M 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:Install 4 Packages 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 99 M 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 122 M 2024-02-07T13:22:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-02-07T13:22:09.520 INFO:teuthology.orchestra.run.smithi094.stdout:(1/4): kernel-5.14.0-418.el9.x86_64.rpm 11 MB/s | 6.0 MB 00:00 2024-02-07T13:22:09.689 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-02-07T13:22:09.776 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-07T13:22:09.776 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-07T13:22:09.858 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-02-07T13:22:09.865 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-418.el9 CentOS-BaseOS 6.0 M 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core x86_64 5.14.0-418.el9 CentOS-BaseOS 21 M 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules x86_64 5.14.0-418.el9 CentOS-BaseOS 39 M 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core x86_64 5.14.0-418.el9 CentOS-BaseOS 33 M 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-02-07T13:22:09.866 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-02-07T13:22:09.867 INFO:teuthology.orchestra.run.smithi190.stdout:Install 4 Packages 2024-02-07T13:22:09.867 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:22:09.867 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 99 M 2024-02-07T13:22:09.867 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 122 M 2024-02-07T13:22:09.867 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-02-07T13:22:10.412 INFO:teuthology.orchestra.run.smithi094.stdout:(2/4): kernel-core-5.14.0-418.el9.x86_64.rpm 14 MB/s | 21 MB 00:01 2024-02-07T13:22:10.454 INFO:teuthology.orchestra.run.smithi190.stdout:(1/4): kernel-5.14.0-418.el9.x86_64.rpm 10 MB/s | 6.0 MB 00:00 2024-02-07T13:22:11.254 INFO:teuthology.orchestra.run.smithi190.stdout:(2/4): kernel-core-5.14.0-418.el9.x86_64.rpm 15 MB/s | 21 MB 00:01 2024-02-07T13:22:11.878 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-07T13:22:11.879 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-07T13:22:11.978 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-418.el9 CentOS-BaseOS 6.0 M 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core x86_64 5.14.0-418.el9 CentOS-BaseOS 21 M 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules x86_64 5.14.0-418.el9 CentOS-BaseOS 39 M 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core x86_64 5.14.0-418.el9 CentOS-BaseOS 33 M 2024-02-07T13:22:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:22:11.989 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-02-07T13:22:11.989 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-02-07T13:22:11.989 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-02-07T13:22:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:22:11.989 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 99 M 2024-02-07T13:22:11.989 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 122 M 2024-02-07T13:22:11.989 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-02-07T13:22:12.005 INFO:teuthology.orchestra.run.smithi190.stdout:(3/4): kernel-modules-5.14.0-418.el9.x86_64.rpm 18 MB/s | 39 MB 00:02 2024-02-07T13:22:12.012 INFO:teuthology.orchestra.run.smithi094.stdout:(3/4): kernel-modules-core-5.14.0-418.el9.x86_6 13 MB/s | 33 MB 00:02 2024-02-07T13:22:12.440 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): kernel-5.14.0-418.el9.x86_64.rpm 13 MB/s | 6.0 MB 00:00 2024-02-07T13:22:12.446 INFO:teuthology.orchestra.run.smithi094.stdout:(4/4): kernel-modules-5.14.0-418.el9.x86_64.rpm 11 MB/s | 39 MB 00:03 2024-02-07T13:22:12.446 INFO:teuthology.orchestra.run.smithi190.stdout:(4/4): kernel-modules-core-5.14.0-418.el9.x86_6 17 MB/s | 33 MB 00:01 2024-02-07T13:22:12.449 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:22:12.449 INFO:teuthology.orchestra.run.smithi190.stdout:Total 38 MB/s | 99 MB 00:02 2024-02-07T13:22:12.450 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-02-07T13:22:12.450 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:22:12.450 INFO:teuthology.orchestra.run.smithi094.stdout:Total 28 MB/s | 99 MB 00:03 2024-02-07T13:22:12.451 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-02-07T13:22:12.821 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-02-07T13:22:12.821 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-02-07T13:22:12.823 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-02-07T13:22:12.823 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-02-07T13:22:12.924 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): kernel-core-5.14.0-418.el9.x86_64.rpm 22 MB/s | 21 MB 00:00 2024-02-07T13:22:13.591 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): kernel-modules-5.14.0-418.el9.x86_64.rpm 24 MB/s | 39 MB 00:01 2024-02-07T13:22:13.855 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-02-07T13:22:13.856 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-02-07T13:22:13.872 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-02-07T13:22:13.873 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-02-07T13:22:13.957 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): kernel-modules-core-5.14.0-418.el9.x86_6 22 MB/s | 33 MB 00:01 2024-02-07T13:22:13.960 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:22:13.961 INFO:teuthology.orchestra.run.smithi138.stdout:Total 50 MB/s | 99 MB 00:01 2024-02-07T13:22:13.961 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-02-07T13:22:14.318 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-02-07T13:22:14.318 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-02-07T13:22:15.346 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-02-07T13:22:15.347 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-02-07T13:22:16.254 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-02-07T13:22:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-02-07T13:22:16.923 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-core-5.14.0-418.el9.x86_64 1/4 2024-02-07T13:22:16.928 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-core-5.14.0-418.el9.x86_64 1/4 2024-02-07T13:22:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T13:22:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T13:22:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-02-07T13:22:18.447 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-core-5.14.0-418.el9.x86_64 1/4 2024-02-07T13:22:18.587 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T13:22:19.803 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T13:22:19.808 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T13:22:19.991 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T13:22:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T13:22:21.325 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T13:22:21.572 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T13:22:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T13:22:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T13:22:24.828 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:26.288 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T13:22:26.493 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:29.576 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:29.588 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:31.197 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:58.612 INFO:teuthology.orchestra.run.smithi190.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-07T13:22:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:22:58.682 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:58.779 INFO:teuthology.orchestra.run.smithi094.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-07T13:22:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:22:58.849 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:59.061 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:59.062 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-418.el9.x86_64 1/4 2024-02-07T13:22:59.062 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T13:22:59.062 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T13:22:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-418.el9.x86_64 1/4 2024-02-07T13:22:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T13:22:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T13:22:59.467 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:59.467 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:22:59.467 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-02-07T13:22:59.468 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-5.14.0-418.el9.x86_64 2024-02-07T13:22:59.468 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core-5.14.0-418.el9.x86_64 2024-02-07T13:22:59.468 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-5.14.0-418.el9.x86_64 2024-02-07T13:22:59.468 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core-5.14.0-418.el9.x86_64 2024-02-07T13:22:59.468 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:22:59.468 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-02-07T13:22:59.756 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:22:59.756 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:22:59.756 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-02-07T13:22:59.756 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-5.14.0-418.el9.x86_64 2024-02-07T13:22:59.756 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core-5.14.0-418.el9.x86_64 2024-02-07T13:22:59.756 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-5.14.0-418.el9.x86_64 2024-02-07T13:22:59.756 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core-5.14.0-418.el9.x86_64 2024-02-07T13:22:59.756 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:22:59.756 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-02-07T13:23:00.001 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T13:23:00.137 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T13:23:00.137 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T13:23:00.137 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-418.el9.x86_64 2024-02-07T13:23:00.137 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-07T13:23:00.137 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-07T13:23:00.138 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-02-07T13:23:00.138 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-02-07T13:23:00.138 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T13:23:00.245 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T13:23:00.246 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T13:23:00.246 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2024-02-07T13:23:00.257 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.06 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 68.el9 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Group : Unspecified 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 8264254 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-07T13:23:00.280 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : builder@centos.org 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : CentOS 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:23:00.281 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2024-02-07T13:23:00.282 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-07T13:23:00.282 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 5.14.0-418.el9.x86_64 2024-02-07T13:23:00.282 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-07T13:23:00.384 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T13:23:00.384 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T13:23:00.384 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-418.el9.x86_64 2024-02-07T13:23:00.384 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-07T13:23:00.384 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-07T13:23:00.384 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-07T13:23:00.385 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=None, version=distro) 2024-02-07T13:23:00.385 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T13:23:00.491 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T13:23:00.492 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T13:23:00.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 68.el9 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8264254 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-07T13:23:00.526 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:23:00.527 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-02-07T13:23:00.528 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-07T13:23:00.528 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 5.14.0-418.el9.x86_64 2024-02-07T13:23:00.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-07T13:23:00.685 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:23:00.685 INFO:teuthology.orchestra.run.smithi138.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-07T13:23:00.685 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:23:00.764 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:23:00.847 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-02-07T13:23:01.120 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-02-07T13:23:01.186 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:23:01.186 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-418.el9.x86_64 1/4 2024-02-07T13:23:01.186 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-core-5.14.0-418.el9.x86_64 2/4 2024-02-07T13:23:01.186 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-5.14.0-418.el9.x86_64 3/4 2024-02-07T13:23:01.640 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-core-5.14.0-418.el9.x86_64 4/4 2024-02-07T13:23:01.640 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:23:01.640 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-02-07T13:23:01.640 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-5.14.0-418.el9.x86_64 2024-02-07T13:23:01.640 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core-5.14.0-418.el9.x86_64 2024-02-07T13:23:01.640 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-5.14.0-418.el9.x86_64 2024-02-07T13:23:01.640 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core-5.14.0-418.el9.x86_64 2024-02-07T13:23:01.641 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:23:01.641 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-02-07T13:23:02.185 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T13:23:02.259 INFO:teuthology.orchestra.run.smithi190.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-07T13:23:02.294 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-02-07T13:23:02.296 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-02-07T13:23:02.311 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.aakXklXthN 2024-02-07T13:23:02.312 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aakXklXthN 2024-02-07T13:23:02.313 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T13:23:02.313 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T13:23:02.314 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-418.el9.x86_64 2024-02-07T13:23:02.314 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-07T13:23:02.314 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-07T13:23:02.314 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-07T13:23:02.314 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2024-02-07T13:23:02.314 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T13:23:02.381 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.aakXklXthN 2024-02-07T13:23:02.423 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-418.el9.x86_64 2024-02-07T13:23:02.423 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-418.el9.x86_64 2024-02-07T13:23:02.423 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-02-07T13:23:02.466 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-02-07T13:23:02.466 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-02-07T13:23:02.466 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-02-07T13:23:02.466 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 68.el9 2024-02-07T13:23:02.466 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-02-07T13:23:02.466 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8264254 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-02-07T13:23:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:23:02.468 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-07T13:23:02.468 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-07T13:23:02.468 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-07T13:23:02.468 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-02-07T13:23:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:23:02.468 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-02-07T13:23:02.469 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-07T13:23:02.469 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 5.14.0-418.el9.x86_64 2024-02-07T13:23:02.469 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-07T13:23:02.522 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.aakXklXthN is 6KB 2024-02-07T13:23:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-07T13:23:02.542 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.aakXklXthN 2024-02-07T13:23:02.553 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-02-07T13:23:02.556 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2024-02-07T13:23:02.557 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-02-07T13:23:02.572 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.Hc3ewyKZAG 2024-02-07T13:23:02.573 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Hc3ewyKZAG 2024-02-07T13:23:02.624 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-07T13:23:02.624 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-07T13:23:02.624 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-07T13:23:02.624 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64.conf 2024-02-07T13:23:02.625 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64 2024-02-07T13:23:02.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.Hc3ewyKZAG 2024-02-07T13:23:02.782 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.Hc3ewyKZAG is 6KB 2024-02-07T13:23:02.788 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-02-07T13:23:02.832 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.Hc3ewyKZAG 2024-02-07T13:23:02.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-02-07T13:23:02.911 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-07T13:23:02.911 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-07T13:23:02.911 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-07T13:23:02.911 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64.conf 2024-02-07T13:23:02.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64 2024-02-07T13:23:03.054 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-02-07T13:23:03.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-02-07T13:23:04.454 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-07T13:23:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-02-07T13:23:04.496 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-02-07T13:23:04.511 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.b4mdDVd5Tx 2024-02-07T13:23:04.511 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.b4mdDVd5Tx 2024-02-07T13:23:04.580 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.b4mdDVd5Tx 2024-02-07T13:23:04.715 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.b4mdDVd5Tx is 6KB 2024-02-07T13:23:04.764 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.b4mdDVd5Tx 2024-02-07T13:23:04.778 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-02-07T13:23:04.843 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-07T13:23:04.843 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-07T13:23:04.843 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-07T13:23:04.843 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64.conf 2024-02-07T13:23:04.844 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-418.el9.x86_64 2024-02-07T13:23:04.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-02-07T13:23:32.817 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-02-07T13:23:32.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-02-07T13:23:32.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:23:33.067 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-07T13:23:33.068 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-07T13:23:33.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:23:35.000 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-07T13:23:35.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-02-07T13:23:35.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:23:51.228 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-02-07T13:23:51.488 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-07T13:23:53.532 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-02-07T13:24:00.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-02-07T13:24:00.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:24:00.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-07T13:24:00.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:24:02.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-02-07T13:24:02.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:24:03.292 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-02-07T13:24:05.596 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-02-07T13:24:15.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-02-07T13:24:15.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:24:17.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-02-07T13:24:17.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:24:33.724 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-02-07T13:24:36.028 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-02-07T13:24:48.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-02-07T13:24:48.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:24:49.096 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-02-07T13:24:49.494 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-02-07T13:24:49.494 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-418.el9.x86_64"... 2024-02-07T13:24:49.495 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-02-07T13:24:49.508 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-418.el9.x86_64 2024-02-07T13:24:49.509 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-418.el9.x86_64 vs 5.14.0-418.el9.x86_64 2024-02-07T13:24:49.509 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-07T13:24:49.509 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-07T13:24:50.509 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-02-07T13:24:50.510 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-07T13:24:50.572 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-02-07T13:24:50.612 DEBUG:teuthology.parallel:result is None 2024-02-07T13:24:51.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-02-07T13:24:51.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:24:51.410 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-02-07T13:24:51.836 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-02-07T13:24:51.836 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-418.el9.x86_64"... 2024-02-07T13:24:51.836 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-02-07T13:24:51.850 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-418.el9.x86_64 2024-02-07T13:24:51.850 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-418.el9.x86_64 vs 5.14.0-418.el9.x86_64 2024-02-07T13:24:51.850 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-07T13:24:51.851 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-07T13:24:52.851 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-07T13:24:52.852 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-07T13:24:52.907 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-02-07T13:24:52.946 DEBUG:teuthology.parallel:result is None 2024-02-07T13:25:00.491 DEBUG:teuthology.orchestra.remote:timed out 2024-02-07T13:25:12.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-07T13:25:12.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:25:12.841 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-02-07T13:25:13.242 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-07T13:25:13.242 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-418.el9.x86_64"... 2024-02-07T13:25:13.242 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-02-07T13:25:13.257 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-418.el9.x86_64 2024-02-07T13:25:13.257 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-418.el9.x86_64 vs 5.14.0-418.el9.x86_64 2024-02-07T13:25:13.257 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-07T13:25:13.258 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-07T13:25:14.259 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-07T13:25:14.259 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-07T13:25:14.356 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-02-07T13:25:14.398 DEBUG:teuthology.parallel:result is None 2024-02-07T13:25:14.398 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-07T13:25:14.406 INFO:teuthology.task.internal:Creating test directory... 2024-02-07T13:25:14.406 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-07T13:25:14.408 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-07T13:25:14.411 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-07T13:25:14.429 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-07T13:25:14.446 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-07T13:25:14.495 INFO:teuthology.task.internal:Creating archive directory... 2024-02-07T13:25:14.495 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-07T13:25:14.498 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-07T13:25:14.500 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-07T13:25:14.530 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-07T13:25:14.536 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-07T13:25:14.536 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-07T13:25:14.564 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-07T13:25:14.566 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-07T13:25:14.596 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:25:14.598 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:25:14.606 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:25:14.610 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:25:14.612 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:25:14.620 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:25:14.621 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-07T13:25:14.628 INFO:teuthology.task.internal:Configuring sudo... 2024-02-07T13:25:14.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-07T13:25:14.657 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-07T13:25:14.660 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-07T13:25:14.691 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-07T13:25:14.700 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-07T13:25:14.700 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-07T13:25:14.726 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-07T13:25:14.729 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-07T13:25:14.747 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:25:14.823 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:25:14.870 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:25:14.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:25:14.995 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:25:14.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-07T13:25:15.063 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:25:15.108 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:25:15.178 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:25:15.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:25:15.267 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:25:15.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-07T13:25:15.337 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:25:15.383 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:25:15.439 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:25:15.523 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:25:15.569 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:25:15.570 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-07T13:25:15.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-02-07T13:25:15.642 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-02-07T13:25:15.644 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-02-07T13:25:15.690 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:25:15.697 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:25:15.736 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:25:16.209 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-07T13:25:16.217 INFO:teuthology.task.internal:Starting timer... 2024-02-07T13:25:16.217 INFO:teuthology.run_tasks:Running task pcp... 2024-02-07T13:25:16.293 INFO:teuthology.run_tasks:Running task selinux... 2024-02-07T13:25:16.323 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-07T13:25:16.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-02-07T13:25:16.381 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-02-07T13:25:16.383 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-02-07T13:25:16.442 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-02-07T13:25:16.443 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-02-07T13:25:16.503 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-02-07T13:25:16.505 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-07T13:25:16.505 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-02-07T13:25:16.524 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-02-07T13:25:16.524 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-02-07T13:25:16.554 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-02-07T13:25:16.555 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-02-07T13:25:16.588 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-02-07T13:25:16.588 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-02-07T13:25:16.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:25:16.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:25:16.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:25:16.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:25:16.646 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:25:16.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:25:16.675 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-07T13:25:16.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-02-07T13:25:16.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-02-07T13:25:16.746 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-02-07T13:25:16.781 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-07T13:25:16.792 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-07T13:25:16.798 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-07T13:25:16.940 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-07T13:25:16.959 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-07T13:25:16.961 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi138.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-07T13:29:24.907 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-02-07T13:29:24.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-07T13:29:24.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:29:25.001 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-02-07T13:29:25.082 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-07T13:29:25.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-02-07T13:29:25.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:29:25.171 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-02-07T13:29:25.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-02-07T13:29:25.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-02-07T13:29:25.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:29:25.334 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-02-07T13:29:25.411 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-02-07T13:29:25.411 INFO:teuthology.run_tasks:Running task clock... 2024-02-07T13:29:25.423 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-07T13:29:25.423 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-07T13:29:25.424 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:29:25.427 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-07T13:29:25.427 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:29:25.429 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-07T13:29:25.429 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:29:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-07T13:29:25.458 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-07T13:29:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-07T13:29:25.474 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-07T13:29:25.481 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-07T13:29:25.498 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-07T13:29:25.501 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-02-07T13:29:25.503 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-02-07T13:29:25.514 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-02-07T13:29:25.517 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-02-07T13:29:25.526 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-02-07T13:29:25.530 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-07T13:29:25.533 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-07T13:29:25.540 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-02-07T13:29:25.547 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-07T13:29:25.550 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-07T13:29:25.556 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-07T13:29:25.572 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-07T13:29:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-02-07T13:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.613 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:29:25.617 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:29:25.617 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-02-07T13:29:25.617 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.617 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.617 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.617 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.633 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:29:25.637 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:29:25.637 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-02-07T13:29:25.637 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.637 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.637 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.637 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:29:25.638 INFO:teuthology.run_tasks:Running task pexec... 2024-02-07T13:29:25.647 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-07T13:29:25.648 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi094.front.sepia.ceph.com 2024-02-07T13:29:25.648 DEBUG:teuthology.orchestra.run.smithi094:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-07T13:29:25.648 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi138.front.sepia.ceph.com 2024-02-07T13:29:25.648 DEBUG:teuthology.orchestra.run.smithi138:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-07T13:29:25.649 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi190.front.sepia.ceph.com 2024-02-07T13:29:25.649 DEBUG:teuthology.orchestra.run.smithi190:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-07T13:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:49 ago on Wed 07 Feb 2024 01:28:37 PM UTC. 2024-02-07T13:29:26.250 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:49 ago on Wed 07 Feb 2024 01:28:37 PM UTC. 2024-02-07T13:29:26.266 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:49 ago on Wed 07 Feb 2024 01:28:37 PM UTC. 2024-02-07T13:29:26.388 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-02-07T13:29:26.402 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-02-07T13:29:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-02-07T13:29:26.402 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-02-07T13:29:26.402 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-02-07T13:29:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-07T13:29:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:29:26.402 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-02-07T13:29:26.403 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-02-07T13:29:26.403 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-02-07T13:29:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:29:26.403 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 3.1 M 2024-02-07T13:29:26.403 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 9.8 M 2024-02-07T13:29:26.403 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-02-07T13:29:26.403 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-02-07T13:29:26.410 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-02-07T13:29:26.410 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-02-07T13:29:26.410 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.1 M 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 9.8 M 2024-02-07T13:29:26.411 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-02-07T13:29:26.417 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-02-07T13:29:26.430 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-02-07T13:29:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-02-07T13:29:26.430 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-02-07T13:29:26.430 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-02-07T13:29:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-07T13:29:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:29:26.431 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-02-07T13:29:26.431 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-02-07T13:29:26.431 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-02-07T13:29:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:29:26.431 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 3.1 M 2024-02-07T13:29:26.431 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 9.8 M 2024-02-07T13:29:26.432 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-02-07T13:29:26.612 INFO:teuthology.orchestra.run.smithi094.stdout:runc-1.1.12-1.el9.x86_64.rpm 15 MB/s | 3.1 MB 00:00 2024-02-07T13:29:26.612 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:29:26.613 INFO:teuthology.orchestra.run.smithi094.stdout:Total 15 MB/s | 3.1 MB 00:00 2024-02-07T13:29:26.613 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-02-07T13:29:26.618 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-02-07T13:29:26.618 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-02-07T13:29:26.653 INFO:teuthology.orchestra.run.smithi138.stdout:runc-1.1.12-1.el9.x86_64.rpm 13 MB/s | 3.1 MB 00:00 2024-02-07T13:29:26.653 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:29:26.654 INFO:teuthology.orchestra.run.smithi138.stdout:Total 13 MB/s | 3.1 MB 00:00 2024-02-07T13:29:26.654 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-02-07T13:29:26.659 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-02-07T13:29:26.659 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-02-07T13:29:26.666 INFO:teuthology.orchestra.run.smithi190.stdout:runc-1.1.12-1.el9.x86_64.rpm 13 MB/s | 3.1 MB 00:00 2024-02-07T13:29:26.667 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:29:26.667 INFO:teuthology.orchestra.run.smithi190.stdout:Total 13 MB/s | 3.1 MB 00:00 2024-02-07T13:29:26.668 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-02-07T13:29:26.673 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-02-07T13:29:26.673 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-02-07T13:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-02-07T13:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-02-07T13:29:26.731 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-02-07T13:29:26.731 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-02-07T13:29:26.744 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-02-07T13:29:26.744 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-02-07T13:29:26.894 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-02-07T13:29:26.934 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-02-07T13:29:26.971 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-02-07T13:29:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T13:29:27.023 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T13:29:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T13:29:27.556 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T13:29:27.612 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T13:29:27.653 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T13:29:27.861 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T13:29:27.861 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:29:27.862 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-02-07T13:29:27.862 INFO:teuthology.orchestra.run.smithi094.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-07T13:29:27.862 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:29:27.862 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-02-07T13:29:27.950 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T13:29:27.950 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:29:27.950 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-02-07T13:29:27.951 INFO:teuthology.orchestra.run.smithi138.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-07T13:29:27.951 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-02-07T13:29:27.951 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-02-07T13:29:27.979 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-07T13:29:27.979 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:29:27.979 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-02-07T13:29:27.979 INFO:teuthology.orchestra.run.smithi190.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-07T13:29:27.979 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:29:27.979 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-02-07T13:29:28.108 DEBUG:teuthology.parallel:result is None 2024-02-07T13:29:28.210 DEBUG:teuthology.parallel:result is None 2024-02-07T13:29:28.216 DEBUG:teuthology.parallel:result is None 2024-02-07T13:29:28.216 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-02-07T13:29:28.227 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-02-07T13:29:28.227 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:29:28.227 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:29:28.242 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-07T13:29:28.243 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-02-07T13:29:28.296 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-07T13:29:28.296 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:28.296 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-07T13:29:28.297 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:28.297 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:28.297 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-07 13:29:23.959022061 +0000 2024-02-07T13:29:28.297 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-07 13:29:00.625452083 +0000 2024-02-07T13:29:28.297 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-07 13:29:00.625452083 +0000 2024-02-07T13:29:28.297 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-07 13:29:00.625452083 +0000 2024-02-07T13:29:28.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-07T13:29:28.361 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-07T13:29:28.362 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-07T13:29:28.362 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000142587 s, 3.6 MB/s 2024-02-07T13:29:28.363 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-07T13:29:28.419 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-02-07T13:29:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-07T13:29:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:28.474 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-07T13:29:28.474 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:28.474 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:28.474 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-07 13:29:23.959022061 +0000 2024-02-07T13:29:28.474 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-07 13:29:00.958445948 +0000 2024-02-07T13:29:28.474 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-07 13:29:00.958445948 +0000 2024-02-07T13:29:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-07 13:29:00.958445948 +0000 2024-02-07T13:29:28.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-07T13:29:28.539 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-07T13:29:28.539 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-07T13:29:28.539 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.00012595 s, 4.1 MB/s 2024-02-07T13:29:28.540 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-07T13:29:28.598 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-02-07T13:29:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-07T13:29:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:28.653 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-07T13:29:28.653 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:28.653 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:28.654 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-07 13:29:23.959022061 +0000 2024-02-07T13:29:28.654 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-07 13:29:01.273440146 +0000 2024-02-07T13:29:28.654 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-07 13:29:01.273440146 +0000 2024-02-07T13:29:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-07 13:29:01.273440146 +0000 2024-02-07T13:29:28.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-07T13:29:28.718 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-07T13:29:28.718 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-07T13:29:28.718 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.2856e-05 s, 6.2 MB/s 2024-02-07T13:29:28.719 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-07T13:29:28.775 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-02-07T13:29:28.830 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-07T13:29:28.830 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:28.830 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-07T13:29:28.830 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:28.830 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:28.830 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-07 13:29:23.959022061 +0000 2024-02-07T13:29:28.830 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-07 13:29:01.600434122 +0000 2024-02-07T13:29:28.830 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-07 13:29:01.600434122 +0000 2024-02-07T13:29:28.831 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-07 13:29:01.600434122 +0000 2024-02-07T13:29:28.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-07T13:29:28.896 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-07T13:29:28.896 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-07T13:29:28.896 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.8934e-05 s, 5.2 MB/s 2024-02-07T13:29:28.897 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-07T13:29:28.953 DEBUG:teuthology.orchestra.run.smithi094:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-07T13:29:29.096 INFO:teuthology.orchestra.run.smithi094.stdout:loop 2024-02-07T13:29:29.098 INFO:tasks.nvme_loop:Connecting nvme_loop smithi094:/dev/vg_nvme/lv_1... 2024-02-07T13:29:29.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-07T13:29:29.135 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-02-07T13:29:29.172 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/vg_nvme/lv_11 2024-02-07T13:29:29.201 INFO:tasks.nvme_loop:Connecting nvme_loop smithi094:/dev/vg_nvme/lv_2... 2024-02-07T13:29:29.201 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-07T13:29:29.281 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-02-07T13:29:29.319 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/vg_nvme/lv_21 2024-02-07T13:29:29.347 INFO:tasks.nvme_loop:Connecting nvme_loop smithi094:/dev/vg_nvme/lv_3... 2024-02-07T13:29:29.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-07T13:29:29.425 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-02-07T13:29:29.462 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/vg_nvme/lv_31 2024-02-07T13:29:29.490 INFO:tasks.nvme_loop:Connecting nvme_loop smithi094:/dev/vg_nvme/lv_4... 2024-02-07T13:29:29.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-07T13:29:29.568 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-02-07T13:29:29.606 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/vg_nvme/lv_41 2024-02-07T13:29:29.633 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:29:29.633 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:29:29.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo nvme list 2024-02-07T13:29:29.753 INFO:teuthology.orchestra.run.smithi094.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-07T13:29:29.753 INFO:teuthology.orchestra.run.smithi094.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-07T13:29:29.754 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/nvme0n1 PHMB7513001L480DGN INTEL SSDPED1D480GA 1 480.10 GB / 480.10 GB 512 B + 0 B E2010325 2024-02-07T13:29:29.754 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/nvme1n1 f8d5754f0aee206d036f Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:29.754 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/nvme2n1 0d5dcead4e7bbc2f2a0d Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:29.754 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/nvme3n1 2b13d56059c566e9829b Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:29.754 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/nvme4n1 bdf1def68e01760597d2 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:29.755 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:29:29.755 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:29:29.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/scratch_devs 2024-02-07T13:29:29.822 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:29:29.822 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:29:29.841 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-07T13:29:29.841 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-02-07T13:29:29.898 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-07T13:29:29.898 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:29.898 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-07T13:29:29.898 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:29.898 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:29.898 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-02-07 13:29:23.931432152 +0000 2024-02-07T13:29:29.899 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-02-07 13:29:01.931913632 +0000 2024-02-07T13:29:29.899 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-02-07 13:29:01.931913632 +0000 2024-02-07T13:29:29.899 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-02-07 13:29:01.931913632 +0000 2024-02-07T13:29:29.899 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-07T13:29:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-02-07T13:29:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-02-07T13:29:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000216027 s, 2.4 MB/s 2024-02-07T13:29:29.969 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-07T13:29:30.027 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-02-07T13:29:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-07T13:29:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:30.083 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-07T13:29:30.083 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:30.083 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:30.083 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-02-07 13:29:23.931432152 +0000 2024-02-07T13:29:30.083 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-02-07 13:29:02.272906178 +0000 2024-02-07T13:29:30.083 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-02-07 13:29:02.272906178 +0000 2024-02-07T13:29:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-02-07 13:29:02.272906178 +0000 2024-02-07T13:29:30.084 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-07T13:29:30.150 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-02-07T13:29:30.150 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-02-07T13:29:30.150 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000192601 s, 2.7 MB/s 2024-02-07T13:29:30.151 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-07T13:29:30.208 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-02-07T13:29:30.264 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-07T13:29:30.264 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:30.264 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-07T13:29:30.264 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:30.265 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:30.265 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-02-07 13:29:23.931432152 +0000 2024-02-07T13:29:30.265 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-02-07 13:29:02.615898680 +0000 2024-02-07T13:29:30.265 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-02-07 13:29:02.615898680 +0000 2024-02-07T13:29:30.265 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-02-07 13:29:02.615898680 +0000 2024-02-07T13:29:30.265 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-07T13:29:30.332 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-02-07T13:29:30.332 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-02-07T13:29:30.332 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000195895 s, 2.6 MB/s 2024-02-07T13:29:30.333 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-07T13:29:30.390 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-02-07T13:29:30.446 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-07T13:29:30.446 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:30.446 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-07T13:29:30.446 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:30.446 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:30.446 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-02-07 13:29:23.931432152 +0000 2024-02-07T13:29:30.446 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-02-07 13:29:02.965891029 +0000 2024-02-07T13:29:30.446 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-02-07 13:29:02.965891029 +0000 2024-02-07T13:29:30.447 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-02-07 13:29:02.965891029 +0000 2024-02-07T13:29:30.447 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-07T13:29:30.513 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-02-07T13:29:30.514 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-02-07T13:29:30.514 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000206276 s, 2.5 MB/s 2024-02-07T13:29:30.514 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-07T13:29:30.570 DEBUG:teuthology.orchestra.run.smithi138:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-07T13:29:30.712 INFO:teuthology.orchestra.run.smithi138.stdout:loop 2024-02-07T13:29:30.713 INFO:tasks.nvme_loop:Connecting nvme_loop smithi138:/dev/vg_nvme/lv_1... 2024-02-07T13:29:30.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-07T13:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-02-07T13:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/vg_nvme/lv_11 2024-02-07T13:29:30.817 INFO:tasks.nvme_loop:Connecting nvme_loop smithi138:/dev/vg_nvme/lv_2... 2024-02-07T13:29:30.817 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-07T13:29:30.896 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-02-07T13:29:30.934 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/vg_nvme/lv_21 2024-02-07T13:29:30.962 INFO:tasks.nvme_loop:Connecting nvme_loop smithi138:/dev/vg_nvme/lv_3... 2024-02-07T13:29:30.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-07T13:29:31.041 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-02-07T13:29:31.078 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/vg_nvme/lv_31 2024-02-07T13:29:31.106 INFO:tasks.nvme_loop:Connecting nvme_loop smithi138:/dev/vg_nvme/lv_4... 2024-02-07T13:29:31.106 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-07T13:29:31.185 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-02-07T13:29:31.222 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/vg_nvme/lv_41 2024-02-07T13:29:31.250 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:29:31.250 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:29:31.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo nvme list 2024-02-07T13:29:31.372 INFO:teuthology.orchestra.run.smithi138.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-07T13:29:31.372 INFO:teuthology.orchestra.run.smithi138.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-07T13:29:31.372 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/nvme0n1 CVFT6161003C400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-07T13:29:31.372 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/nvme1n1 6091bc1dc322f79ae5a6 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:31.372 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/nvme2n1 138f1f7bbf60300bb1e0 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:31.372 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/nvme3n1 58fcba7ac1e6ecacbaa1 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:31.372 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/nvme4n1 e70c19876b5f9eb8f13c Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:31.373 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:29:31.373 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:29:31.373 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/scratch_devs 2024-02-07T13:29:31.441 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:29:31.441 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:29:31.460 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-07T13:29:31.461 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-02-07T13:29:31.518 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-07T13:29:31.518 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:31.518 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-02-07T13:29:31.518 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:31.518 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:31.518 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-02-07 13:29:23.942338672 +0000 2024-02-07T13:29:31.519 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-02-07 13:29:02.073983367 +0000 2024-02-07T13:29:31.519 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-02-07 13:29:02.073983367 +0000 2024-02-07T13:29:31.519 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-02-07 13:29:02.073983367 +0000 2024-02-07T13:29:31.519 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-07T13:29:31.588 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-02-07T13:29:31.588 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-02-07T13:29:31.588 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000223446 s, 2.3 MB/s 2024-02-07T13:29:31.590 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-07T13:29:31.647 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-02-07T13:29:31.705 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-07T13:29:31.705 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:31.705 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-07T13:29:31.705 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:31.705 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:31.705 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-02-07 13:29:23.942338672 +0000 2024-02-07T13:29:31.705 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-02-07 13:29:02.405973549 +0000 2024-02-07T13:29:31.705 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-02-07 13:29:02.405973549 +0000 2024-02-07T13:29:31.705 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-02-07 13:29:02.405973549 +0000 2024-02-07T13:29:31.706 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-07T13:29:31.774 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-02-07T13:29:31.774 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-02-07T13:29:31.774 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000321448 s, 1.6 MB/s 2024-02-07T13:29:31.775 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-07T13:29:31.832 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-02-07T13:29:31.889 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-07T13:29:31.889 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:31.890 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-07T13:29:31.890 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:31.890 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:31.890 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-02-07 13:29:23.942338672 +0000 2024-02-07T13:29:31.890 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-02-07 13:29:02.735963791 +0000 2024-02-07T13:29:31.890 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-02-07 13:29:02.735963791 +0000 2024-02-07T13:29:31.890 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-02-07 13:29:02.735963791 +0000 2024-02-07T13:29:31.890 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-07T13:29:31.958 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-02-07T13:29:31.958 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-02-07T13:29:31.958 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000167579 s, 3.1 MB/s 2024-02-07T13:29:31.959 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-07T13:29:32.017 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-02-07T13:29:32.073 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-07T13:29:32.074 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:29:32.074 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-07T13:29:32.074 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:29:32.074 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:29:32.074 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-02-07 13:29:23.942338672 +0000 2024-02-07T13:29:32.074 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-02-07 13:29:03.073953796 +0000 2024-02-07T13:29:32.074 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-02-07 13:29:03.073953796 +0000 2024-02-07T13:29:32.074 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-02-07 13:29:03.073953796 +0000 2024-02-07T13:29:32.075 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-07T13:29:32.140 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-02-07T13:29:32.141 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-02-07T13:29:32.141 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000183422 s, 2.8 MB/s 2024-02-07T13:29:32.142 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-07T13:29:32.198 DEBUG:teuthology.orchestra.run.smithi190:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-07T13:29:32.343 INFO:teuthology.orchestra.run.smithi190.stdout:loop 2024-02-07T13:29:32.344 INFO:tasks.nvme_loop:Connecting nvme_loop smithi190:/dev/vg_nvme/lv_1... 2024-02-07T13:29:32.345 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-07T13:29:32.382 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-02-07T13:29:32.419 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/vg_nvme/lv_11 2024-02-07T13:29:32.457 INFO:tasks.nvme_loop:Connecting nvme_loop smithi190:/dev/vg_nvme/lv_2... 2024-02-07T13:29:32.457 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-07T13:29:32.533 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-02-07T13:29:32.571 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/vg_nvme/lv_21 2024-02-07T13:29:32.599 INFO:tasks.nvme_loop:Connecting nvme_loop smithi190:/dev/vg_nvme/lv_3... 2024-02-07T13:29:32.599 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-07T13:29:32.677 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-02-07T13:29:32.714 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/vg_nvme/lv_31 2024-02-07T13:29:32.742 INFO:tasks.nvme_loop:Connecting nvme_loop smithi190:/dev/vg_nvme/lv_4... 2024-02-07T13:29:32.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-07T13:29:32.820 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-02-07T13:29:32.857 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/vg_nvme/lv_41 2024-02-07T13:29:32.885 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:29:32.886 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:29:32.940 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme list 2024-02-07T13:29:33.006 INFO:teuthology.orchestra.run.smithi190.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-07T13:29:33.006 INFO:teuthology.orchestra.run.smithi190.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-07T13:29:33.006 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/nvme0n1 CVFT63210012400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-07T13:29:33.006 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/nvme1n1 9cbc534b67f706b67652 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:33.006 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/nvme2n1 cccddb2c9e925480777e Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:33.006 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/nvme3n1 56b8d14c79c131392f9f Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:33.006 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/nvme4n1 8c76b88805ed3432cc76 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:29:33.007 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:29:33.007 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:29:33.007 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/scratch_devs 2024-02-07T13:29:33.074 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-07T13:29:33.172 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '44458ccedb18241cc2e17f10cdab3d1385399261'} 2024-02-07T13:29:33.172 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:29:33.172 INFO:tasks.cephadm:Cluster fsid is edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:29:33.173 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-07T13:29:33.173 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.94', 'mon.b': '172.21.15.138', 'mon.c': '172.21.15.190'} 2024-02-07T13:29:33.173 INFO:tasks.cephadm:First mon is mon.a on smithi094 2024-02-07T13:29:33.173 INFO:tasks.cephadm:First mgr is y 2024-02-07T13:29:33.173 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-07T13:29:33.173 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-02-07T13:29:33.205 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-02-07T13:29:33.238 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2024-02-07T13:29:33.265 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-07T13:29:33.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:29:33.411 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '44458ccedb18241cc2e17f10cdab3d1385399261', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76537/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1252-g44458cce', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1252.g44458cce'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-05 21:02:35.601617', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-testing-4-2024-02-05-1854', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-07T13:29:33.548 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-guits-testing-4-2024-02-05-1854, sha1 44458ccedb18241cc2e17f10cdab3d1385399261 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:29:33.549 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm 2024-02-07T13:29:33.549 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-07T13:29:34.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 766878 Feb 7 13:29 /home/ubuntu/cephtest/cephadm 2024-02-07T13:29:34.061 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-07T13:29:34.541 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 766878 Feb 7 13:29 /home/ubuntu/cephtest/cephadm 2024-02-07T13:29:34.541 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-07T13:29:35.248 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r--. 1 ubuntu ubuntu 766878 Feb 7 13:29 /home/ubuntu/cephtest/cephadm 2024-02-07T13:29:35.249 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-07T13:29:35.267 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-07T13:29:35.286 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-07T13:29:35.318 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 on all hosts... 2024-02-07T13:29:35.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 pull 2024-02-07T13:29:35.321 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 pull 2024-02-07T13:29:35.329 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 pull 2024-02-07T13:29:36.127 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:29:36.147 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:29:36.164 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:30:10.671 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-02-07T13:30:10.671 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev)", 2024-02-07T13:30:10.671 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f", 2024-02-07T13:30:10.671 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2024-02-07T13:30:10.671 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe" 2024-02-07T13:30:10.672 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-02-07T13:30:10.672 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-02-07T13:30:21.768 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-02-07T13:30:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev)", 2024-02-07T13:30:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f", 2024-02-07T13:30:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-02-07T13:30:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe" 2024-02-07T13:30:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-02-07T13:30:21.768 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-02-07T13:30:23.757 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-07T13:30:23.757 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev)", 2024-02-07T13:30:23.757 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f", 2024-02-07T13:30:23.757 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-02-07T13:30:23.757 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe" 2024-02-07T13:30:23.757 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-07T13:30:23.757 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-07T13:30:23.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-02-07T13:30:23.847 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-02-07T13:30:23.883 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2024-02-07T13:30:23.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-02-07T13:30:23.945 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-02-07T13:30:23.974 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2024-02-07T13:30:24.006 INFO:tasks.cephadm:Writing seed config... 2024-02-07T13:30:24.007 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-07T13:30:24.007 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-07T13:30:24.007 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-02-07T13:30:24.007 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-02-07T13:30:24.007 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-07T13:30:24.007 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-07T13:30:24.007 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-07T13:30:24.007 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-07T13:30:24.008 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-07T13:30:24.008 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-07T13:30:24.008 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:30:24.008 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-07T13:30:24.023 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = edf33ada-c5bc-11ee-95b6-87774f69a715 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-02-07T13:30:24.023 DEBUG:teuthology.orchestra.run.smithi094:mon.a> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.a.service 2024-02-07T13:30:24.066 DEBUG:teuthology.orchestra.run.smithi094:mgr.y> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.y.service 2024-02-07T13:30:24.109 INFO:tasks.cephadm:Bootstrapping... 2024-02-07T13:30:24.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 -v bootstrap --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.94 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:30:24.298 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:30:24.299 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261', '-v', 'bootstrap', '--fsid', 'edf33ada-c5bc-11ee-95b6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.94', '--skip-admin-label'] 2024-02-07T13:30:24.323 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-07T13:30:24.323 INFO:teuthology.orchestra.run.smithi094.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-07T13:30:24.324 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying podman|docker is present... 2024-02-07T13:30:24.346 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-07T13:30:24.346 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying lvm2 is present... 2024-02-07T13:30:24.347 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying time synchronization is in place... 2024-02-07T13:30:24.353 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-07T13:30:24.353 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-07T13:30:24.359 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-07T13:30:24.359 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-07T13:30:24.365 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-02-07T13:30:24.371 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-02-07T13:30:24.371 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-02-07T13:30:24.371 INFO:teuthology.orchestra.run.smithi094.stdout:Repeating the final host check... 2024-02-07T13:30:24.395 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-07T13:30:24.395 INFO:teuthology.orchestra.run.smithi094.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-07T13:30:24.395 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl is present 2024-02-07T13:30:24.395 INFO:teuthology.orchestra.run.smithi094.stdout:lvcreate is present 2024-02-07T13:30:24.401 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-07T13:30:24.401 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-07T13:30:24.406 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-07T13:30:24.407 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-07T13:30:24.413 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-02-07T13:30:24.419 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-02-07T13:30:24.419 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-02-07T13:30:24.419 INFO:teuthology.orchestra.run.smithi094.stdout:Host looks OK 2024-02-07T13:30:24.419 INFO:teuthology.orchestra.run.smithi094.stdout:Cluster fsid: edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:30:24.419 INFO:teuthology.orchestra.run.smithi094.stdout:Acquiring lock 139853951534128 on /run/cephadm/edf33ada-c5bc-11ee-95b6-87774f69a715.lock 2024-02-07T13:30:24.419 INFO:teuthology.orchestra.run.smithi094.stdout:Lock 139853951534128 acquired on /run/cephadm/edf33ada-c5bc-11ee-95b6-87774f69a715.lock 2024-02-07T13:30:24.419 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 3300 ... 2024-02-07T13:30:24.420 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 6789 ... 2024-02-07T13:30:24.420 INFO:teuthology.orchestra.run.smithi094.stdout:Base mon IP(s) is [172.21.15.94:3300, 172.21.15.94:6789], mon addrv is [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-02-07T13:30:24.422 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.94 metric 100 2024-02-07T13:30:24.423 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.94 metric 100 2024-02-07T13:30:24.425 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-07T13:30:24.425 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-07T13:30:24.425 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2024-02-07T13:30:24.427 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-07T13:30:24.428 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-07T13:30:24.428 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-07T13:30:24.428 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-07T13:30:24.428 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-02-07T13:30:24.428 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-07T13:30:24.429 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-02-07T13:30:24.429 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-02-07T13:30:24.429 INFO:teuthology.orchestra.run.smithi094.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-07T13:30:24.429 INFO:teuthology.orchestra.run.smithi094.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-07T13:30:24.429 INFO:teuthology.orchestra.run.smithi094.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:30:24.885 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f 2024-02-07T13:30:24.885 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:30:24.885 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Getting image source signatures 2024-02-07T13:30:24.885 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:a997f0e9fc346761d24d2a6e2e6c8569e51b56d4493dc8639d7f291d92935b5f 2024-02-07T13:30:24.885 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-07T13:30:24.885 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying config sha256:79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f 2024-02-07T13:30:24.885 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-07T13:30:25.873 INFO:teuthology.orchestra.run.smithi094.stdout:ceph: stdout ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev) 2024-02-07T13:30:25.873 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph version: ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev) 2024-02-07T13:30:25.873 INFO:teuthology.orchestra.run.smithi094.stdout:Extracting ceph user uid/gid from container image... 2024-02-07T13:30:26.883 INFO:teuthology.orchestra.run.smithi094.stdout:stat: stdout 167 167 2024-02-07T13:30:26.883 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial keys... 2024-02-07T13:30:28.056 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQDzhcNlX2jXEBAApc4IN568eZ3fKXbPkTbUJw== 2024-02-07T13:30:28.994 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQD0hcNlx5dYGhAAeZHV5OlYO2VDRULgfeip4g== 2024-02-07T13:30:30.050 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQD1hcNl/hcuFRAAGUZv0MqOY3A9tcdSkAahEQ== 2024-02-07T13:30:30.050 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial monmap... 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool for a [v2:172.21.15.94:3300,v1:172.21.15.94:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: set fsid to edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:30:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mon... 2024-02-07T13:30:31.904 INFO:teuthology.orchestra.run.smithi094.stdout:create mon.a on 2024-02-07T13:30:32.249 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-07T13:30:32.436 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715.target. 2024-02-07T13:30:32.436 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715.target. 2024-02-07T13:30:32.687 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.a 2024-02-07T13:30:32.687 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.a.service: Unit ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.a.service not loaded. 2024-02-07T13:30:32.873 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715.target.wants/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@.service. 2024-02-07T13:30:33.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:33 smithi094 podman[25049]: 2024-02-07 13:30:33.112917137 +0000 UTC m=+0.120210665 container create 8f5b1d76244a551d044dcc1bd52940b686730645719f52f6022828e7e817ae59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-07T13:30:33.427 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T13:30:33.427 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-02-07T13:30:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T13:30:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-07T13:30:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-02-07T13:30:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-02-07T13:30:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon to start... 2024-02-07T13:30:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon... 2024-02-07T13:30:33.558 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:33 smithi094 podman[25049]: 2024-02-07 13:30:33.306890101 +0000 UTC m=+0.314183633 container init 8f5b1d76244a551d044dcc1bd52940b686730645719f52f6022828e7e817ae59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T13:30:33.558 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:33 smithi094 podman[25049]: 2024-02-07 13:30:33.311992881 +0000 UTC m=+0.319286415 container start 8f5b1d76244a551d044dcc1bd52940b686730645719f52f6022828e7e817ae59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-07T13:30:33.558 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:33 smithi094 ceph-mon[25086]: mkfs edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:30:33.558 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:33 smithi094 ceph-mon[25086]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-07T13:30:33.559 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:33 smithi094 bash[25049]: 8f5b1d76244a551d044dcc1bd52940b686730645719f52f6022828e7e817ae59 2024-02-07T13:30:33.559 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:33 smithi094 systemd[1]: Started Ceph mon.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:30:34.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:34 smithi094 ceph-mon[25086]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-07T13:30:34.555 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:34 smithi094 ceph-mon[25086]: monmap e1: 1 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:30:34.555 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:34 smithi094 ceph-mon[25086]: fsmap 2024-02-07T13:30:34.555 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:34 smithi094 ceph-mon[25086]: osdmap e1: 0 total, 0 up, 0 in 2024-02-07T13:30:34.555 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:34 smithi094 ceph-mon[25086]: mgrmap e1: no daemons active 2024-02-07T13:30:34.555 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:34 smithi094 ceph-mon[25086]: from='client.? 172.21.15.94:0/206356801' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout cluster: 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout id: edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout services: 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.598947s) 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout data: 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-07T13:30:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-07T13:30:34.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pgs: 2024-02-07T13:30:34.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:34.560 INFO:teuthology.orchestra.run.smithi094.stdout:mon is available 2024-02-07T13:30:34.560 INFO:teuthology.orchestra.run.smithi094.stdout:Assimilating anything we can from ceph.conf... 2024-02-07T13:30:35.619 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:35 smithi094 ceph-mon[25086]: from='client.? 172.21.15.94:0/2312229299' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-07T13:30:35.619 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:35 smithi094 ceph-mon[25086]: from='client.? 172.21.15.94:0/2312229299' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-07T13:30:35.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-02-07T13:30:35.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-07T13:30:35.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-07T13:30:35.624 INFO:teuthology.orchestra.run.smithi094.stdout:Generating new minimal ceph.conf... 2024-02-07T13:30:36.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:36 smithi094 ceph-mon[25086]: from='client.? 172.21.15.94:0/2300138077' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:30:36.711 INFO:teuthology.orchestra.run.smithi094.stdout:Restarting the monitor... 2024-02-07T13:30:36.990 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:36 smithi094 systemd[1]: Stopping Ceph mon.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:30:37.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:36 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a[25063]: 2024-02-07T13:30:36.988+0000 7f14987ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:30:37.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:36 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a[25063]: 2024-02-07T13:30:36.988+0000 7f14987ad700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-07T13:30:37.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:37 smithi094 podman[25420]: 2024-02-07 13:30:37.133101545 +0000 UTC m=+0.257022645 container died 8f5b1d76244a551d044dcc1bd52940b686730645719f52f6022828e7e817ae59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T13:30:37.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:37 smithi094 podman[25420]: 2024-02-07 13:30:37.46203784 +0000 UTC m=+0.585958939 container cleanup 8f5b1d76244a551d044dcc1bd52940b686730645719f52f6022828e7e817ae59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-02-07T13:30:37.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:37 smithi094 bash[25420]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a 2024-02-07T13:30:37.755 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:37 smithi094 podman[25439]: 2024-02-07 13:30:37.629786094 +0000 UTC m=+0.500380357 container remove 8f5b1d76244a551d044dcc1bd52940b686730645719f52f6022828e7e817ae59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T13:30:38.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:37 smithi094 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-07T13:30:38.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:37 smithi094 systemd[1]: Stopped Ceph mon.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:30:38.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:37 smithi094 systemd[1]: Starting Ceph mon.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:30:38.371 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 podman[25538]: 2024-02-07 13:30:37.978231958 +0000 UTC m=+0.017313236 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:30:38.371 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 podman[25538]: 2024-02-07 13:30:38.080709576 +0000 UTC m=+0.119790883 container create 7b74c730dca77210376dcd0f0f19fdf21895bc0de2635d878720bfafd36da988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True) 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 podman[25538]: 2024-02-07 13:30:38.267153569 +0000 UTC m=+0.306234845 container init 7b74c730dca77210376dcd0f0f19fdf21895bc0de2635d878720bfafd36da988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 podman[25538]: 2024-02-07 13:30:38.272792472 +0000 UTC m=+0.311873740 container start 7b74c730dca77210376dcd0f0f19fdf21895bc0de2635d878720bfafd36da988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: set uid:gid to 167:167 (ceph:ceph) 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev), process ceph-mon, pid 7 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: pidfile_write: ignore empty --pid-file 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: load: jerasure load: lrc 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: RocksDB version: 7.9.2 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Git sha 0 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Compile date 2024-02-05 19:41:51 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: DB SUMMARY 2024-02-07T13:30:38.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: DB Session ID: 5PET1UJFFILTKMUV6527 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: CURRENT file: CURRENT 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: IDENTITY file: IDENTITY 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 84623 ; 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.error_if_exists: 0 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.create_if_missing: 0 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.paranoid_checks: 1 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-07T13:30:38.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.env: 0x5635c5f47ae0 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.fs: PosixFileSystem 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.info_log: 0x5635c6f458c0 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_file_opening_threads: 16 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.statistics: (nil) 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.use_fsync: 0 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_log_file_size: 0 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.keep_log_file_num: 1000 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.recycle_log_file_num: 0 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.allow_fallocate: 1 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.allow_mmap_reads: 0 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.allow_mmap_writes: 0 2024-02-07T13:30:38.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.use_direct_reads: 0 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.create_missing_column_families: 0 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.db_log_dir: 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.wal_dir: 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.advise_random_on_open: 1 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.db_write_buffer_size: 0 2024-02-07T13:30:38.375 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.write_buffer_manager: 0x5635c6f19400 2024-02-07T13:30:38.376 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-07T13:30:38.376 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-07T13:30:38.376 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-07T13:30:38.376 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.rate_limiter: (nil) 2024-02-07T13:30:38.376 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-07T13:30:38.377 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.wal_recovery_mode: 2 2024-02-07T13:30:38.377 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.enable_thread_tracking: 0 2024-02-07T13:30:38.377 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.enable_pipelined_write: 0 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.unordered_write: 0 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.row_cache: None 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.wal_filter: None 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.allow_ingest_behind: 0 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.two_write_queues: 0 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.manual_wal_flush: 0 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.wal_compression: 0 2024-02-07T13:30:38.378 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.atomic_flush: 0 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.log_readahead_size: 0 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.best_efforts_recovery: 0 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.allow_data_in_errors: 0 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.db_host_id: __hostname__ 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-07T13:30:38.379 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_background_jobs: 2 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_background_compactions: -1 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_subcompactions: 1 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_total_wal_size: 0 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_open_files: -1 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bytes_per_sync: 0 2024-02-07T13:30:38.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_readahead_size: 0 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_background_flushes: -1 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Compression algorithms supported: 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: kZSTD supported: 0 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: kXpressCompression supported: 0 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: kLZ4HCCompression supported: 1 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: kZlibCompression supported: 1 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: kSnappyCompression supported: 1 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: kLZ4Compression supported: 1 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: kBZip2Compression supported: 0 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-07T13:30:38.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.merge_operator: 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_filter: None 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_filter_factory: None 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.sst_partitioner_factory: None 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5635c6f459c0) 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-07T13:30:38.382 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_type: 0 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_index_type: 0 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_shortening: 1 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: checksum: 4 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: no_block_cache: 0 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache: 0x5635c6f49090 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_options: 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: capacity : 536870912 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_shard_bits : 4 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: strict_capacity_limit : 0 2024-02-07T13:30:38.383 INFO:journalctl@ceph.mon.a.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_compressed: (nil) 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: persistent_cache: (nil) 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size: 4096 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size_deviation: 10 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_restart_interval: 16 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_block_restart_interval: 1 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: metadata_block_size: 4096 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: partition_filters: 0 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: use_delta_encoding: 1 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: filter_policy: bloomfilter 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: whole_key_filtering: 1 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: verify_compression: 0 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-02-07T13:30:38.384 INFO:journalctl@ceph.mon.a.smithi094.stdout: format_version: 5 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_index_compression: 1 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_align: 0 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout: max_auto_readahead_size: 262144 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout: prepopulate_block_cache: 0 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout: initial_auto_readahead_size: 8192 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.write_buffer_size: 33554432 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_write_buffer_number: 2 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression: NoCompression 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression: Disabled 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.prefix_extractor: nullptr 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.num_levels: 7 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-07T13:30:38.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression_opts.level: 32767 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression_opts.strategy: 0 2024-02-07T13:30:38.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression_opts.enabled: false 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.target_file_size_base: 67108864 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-07T13:30:38.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.arena_block_size: 1048576 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.disable_auto_compactions: 0 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-07T13:30:38.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.table_properties_collectors: 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.inplace_update_support: 0 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-07T13:30:38.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.bloom_locality: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.max_successive_merges: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.paranoid_file_checks: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.force_consistency_checks: 1 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.report_bg_io_stats: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.ttl: 2592000 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.enable_blob_files: false 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.min_blob_size: 0 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.blob_file_size: 268435456 2024-02-07T13:30:38.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.blob_file_starting_level: 0 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a495e615-72e6-477b-b241-26016b1591bd 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707312638301450, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707312638302318, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81248, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 252, "table_properties": {"data_size": 79435, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 73744, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 211, "num_filter_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707312638, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a495e615-72e6-477b-b241-26016b1591bd", "db_session_id": "5PET1UJFFILTKMUV6527", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707312638302396, "job": 1, "event": "recovery_finished"} 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-07T13:30:38.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5635c703a000 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: DB pointer 0x5635c7024000 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** DB Stats ** 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-07T13:30:38.392 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: L0 2/0 81.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: Sum 2/0 81.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T13:30:38.393 INFO:journalctl@ceph.mon.a.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative compaction: 0.00 GB write, 28.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval compaction: 0.00 GB write, 28.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache BinnedLRUCache@0x5635c6f49090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-07T13:30:38.394 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: starting mon.a rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mon.a@-1(???) e1 preinit fsid edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mon.a@-1(???).mds e1 new map 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mon.a@-1(???).mds e1 print_map 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout: e1 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout: legacy client fscid: -1 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout: No filesystems configured 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-07T13:30:38.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-07T13:30:38.396 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: monmap e1: 1 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:30:38.396 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: fsmap 2024-02-07T13:30:38.396 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: osdmap e1: 0 total, 0 up, 0 in 2024-02-07T13:30:38.396 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 ceph-mon[25576]: mgrmap e1: no daemons active 2024-02-07T13:30:38.396 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 bash[25538]: 7b74c730dca77210376dcd0f0f19fdf21895bc0de2635d878720bfafd36da988 2024-02-07T13:30:38.396 INFO:teuthology.orchestra.run.smithi094.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-07T13:30:38.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:38 smithi094 systemd[1]: Started Ceph mon.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:30:39.366 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-07T13:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mgr... 2024-02-07T13:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-07T13:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-07T13:30:39.640 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.y 2024-02-07T13:30:39.640 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.y.service: Unit ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.y.service not loaded. 2024-02-07T13:30:39.795 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715.target.wants/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@.service. 2024-02-07T13:30:40.148 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:39 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1044521748' entity='client.admin' 2024-02-07T13:30:40.148 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:40 smithi094 podman[25817]: 2024-02-07 13:30:39.932692695 +0000 UTC m=+0.017351105 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T13:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-02-07T13:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T13:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-07T13:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-02-07T13:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-02-07T13:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T13:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-02-07T13:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T13:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-07T13:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-02-07T13:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-07T13:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr to start... 2024-02-07T13:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr... 2024-02-07T13:30:40.510 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:40 smithi094 podman[25817]: 2024-02-07 13:30:40.260199583 +0000 UTC m=+0.344857988 container init 15e2e624d8661c76f4ccd17ffec73a01317f4be9898327f885257ecaabe7658e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.build-date=20240131) 2024-02-07T13:30:40.511 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:40 smithi094 podman[25817]: 2024-02-07 13:30:40.26537351 +0000 UTC m=+0.350031908 container start 15e2e624d8661c76f4ccd17ffec73a01317f4be9898327f885257ecaabe7658e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:30:40.511 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:40 smithi094 bash[25817]: 15e2e624d8661c76f4ccd17ffec73a01317f4be9898327f885257ecaabe7658e 2024-02-07T13:30:40.511 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:40 smithi094 systemd[1]: Started Ceph mgr.y for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:30:41.240 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:41 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1273706155' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T13:30:41.241 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:40 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:40.819+0000 7f1b6391a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:30:41.241 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:40 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:40.926+0000 7f1b6391a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:30:41.471 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:41.471 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:30:41.471 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "edf33ada-c5bc-11ee-95b6-87774f69a715", 2024-02-07T13:30:41.471 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T13:30:41.471 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T13:30:41.471 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T13:30:41.472 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T13:30:41.473 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:41.473 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T13:30:41.473 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:41.473 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T13:30:41.473 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T13:30:41.473 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T13:30:41.473 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T13:30:41.473 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T13:30:41.473 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T13:30:41.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:41.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T13:30:41.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T13:30:41.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T13:30:41.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T13:30:41.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T13:30:41.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T13:30:41.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T13:30:41.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T13:30:33.380370+0000", 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:30:41.478 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (1/15)... 2024-02-07T13:30:41.905 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:41 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:41.581+0000 7f1b6391a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:30:41.905 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:41 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:41.671+0000 7f1b6391a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:30:41.905 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:41 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:41.810+0000 7f1b6391a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:30:42.244 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:41 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:41.903+0000 7f1b6391a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:30:42.244 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:41 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:41.985+0000 7f1b6391a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:30:42.244 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:42 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:42.243+0000 7f1b6391a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:30:42.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:42 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:42.350+0000 7f1b6391a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:30:42.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:42 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:42.434+0000 7f1b6391a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:30:43.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:42 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:42.527+0000 7f1b6391a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:30:43.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:42 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:42.700+0000 7f1b6391a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:30:43.903 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:43 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:43.660+0000 7f1b6391a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:30:43.903 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:43 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:43.794+0000 7f1b6391a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:30:44.228 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:44 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:44.136+0000 7f1b6391a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:30:44.228 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:44 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:44.227+0000 7f1b6391a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:30:44.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:44.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:30:44.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "edf33ada-c5bc-11ee-95b6-87774f69a715", 2024-02-07T13:30:44.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T13:30:44.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T13:30:44.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T13:30:44.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T13:30:44.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:44.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T13:30:44.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:44.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T13:30:44.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T13:30:44.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T13:30:44.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T13:30:44.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T13:30:44.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T13:30:44.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:44.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T13:30:44.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T13:30:44.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T13:30:44.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T13:30:44.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T13:30:44.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T13:30:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T13:30:33.380370+0000", 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:30:44.487 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (2/15)... 2024-02-07T13:30:44.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:44 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1517421763' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T13:30:44.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:44 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:44.422+0000 7f1b6391a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:30:44.884 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:44 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:44.604+0000 7f1b6391a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:30:45.139 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:44 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:44.883+0000 7f1b6391a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:30:45.139 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:45 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:45.138+0000 7f1b6391a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:30:45.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:45 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:45.370+0000 7f1b6391a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:30:46.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:46 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:46.175+0000 7f1b6391a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:30:46.859 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:46 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:46.773+0000 7f1b6391a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:30:47.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:47 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1161011343' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T13:30:47.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:46 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:46.875+0000 7f1b6391a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:30:47.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:47 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:30:47.100+0000 7f1b6391a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:30:47.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:47.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:30:47.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "edf33ada-c5bc-11ee-95b6-87774f69a715", 2024-02-07T13:30:47.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T13:30:47.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T13:30:47.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T13:30:47.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T13:30:47.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T13:30:33.380370+0000", 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:30:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:47.619 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T13:30:47.619 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:30:47.619 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (3/15)... 2024-02-07T13:30:48.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: Activating manager daemon y 2024-02-07T13:30:48.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: mgrmap e2: y(active, starting, since 0.00295312s) 2024-02-07T13:30:48.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:30:48.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:30:48.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:30:48.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:30:48.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:30:48.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: Manager daemon y is now available 2024-02-07T13:30:48.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:30:48.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:30:48.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' 2024-02-07T13:30:48.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' 2024-02-07T13:30:48.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:48 smithi094 ceph-mon[25576]: from='mgr.14100 172.21.15.94:0/3109583149' entity='mgr.y' 2024-02-07T13:30:49.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:49 smithi094 ceph-mon[25576]: mgrmap e3: y(active, since 1.00595s) 2024-02-07T13:30:51.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:51 smithi094 ceph-mon[25576]: mgrmap e4: y(active, since 3s) 2024-02-07T13:30:53.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:53 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3788797399' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "edf33ada-c5bc-11ee-95b6-87774f69a715", 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T13:30:57.506 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T13:30:57.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T13:30:57.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:57.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T13:30:57.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:30:57.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T13:30:33.380370+0000", 2024-02-07T13:30:57.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:30:57.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:30:57.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T13:30:57.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:30:57.514 INFO:teuthology.orchestra.run.smithi094.stdout:mgr is available 2024-02-07T13:30:58.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:58 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/874531316' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-07T13:30:58.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:58 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/874531316' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-07T13:30:58.710 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:58.710 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-02-07T13:30:58.710 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:30:58.710 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-07T13:30:58.710 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-07T13:30:58.710 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-07T13:30:58.710 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-07T13:30:58.710 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:58.711 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-07T13:30:58.711 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-07T13:30:58.711 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-07T13:30:58.711 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-02-07T13:30:58.711 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-07T13:30:58.711 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-07T13:30:58.711 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling cephadm module... 2024-02-07T13:30:59.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:30:59 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/244708452' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-07T13:30:59.764 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:59 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ignoring --setuser ceph since I am not root 2024-02-07T13:30:59.764 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:30:59 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ignoring --setgroup ceph since I am not root 2024-02-07T13:31:00.296 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:00 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:00.040+0000 7fdf25619200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:31:00.296 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:00 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:00.140+0000 7fdf25619200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:31:00.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:00 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/244708452' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-07T13:31:00.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:00 smithi094 ceph-mon[25576]: mgrmap e5: y(active, since 12s) 2024-02-07T13:31:01.037 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:00 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:00.801+0000 7fdf25619200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:31:01.037 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:00 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:00.894+0000 7fdf25619200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:31:01.037 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:01 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:01.036+0000 7fdf25619200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:31:01.476 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:01 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:01.132+0000 7fdf25619200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:31:01.476 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:01 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:01.215+0000 7fdf25619200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:31:01.476 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:01 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:01.476+0000 7fdf25619200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:31:01.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:31:01.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-07T13:31:01.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-07T13:31:01.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-07T13:31:01.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-07T13:31:01.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:31:01.638 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-02-07T13:31:01.638 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 5... 2024-02-07T13:31:01.759 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:01 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/362391232' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-07T13:31:01.759 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:01 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:01.585+0000 7fdf25619200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:31:01.759 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:01 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:01.673+0000 7fdf25619200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:31:02.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:01 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:01.768+0000 7fdf25619200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:31:02.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:01 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:01.946+0000 7fdf25619200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:31:03.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:02 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:02.905+0000 7fdf25619200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:31:03.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:03 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:03.035+0000 7fdf25619200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:31:03.666 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:03 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:03.384+0000 7fdf25619200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:31:03.666 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:03 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:03.473+0000 7fdf25619200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:31:03.666 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:03 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:03.666+0000 7fdf25619200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:31:04.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:03 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:03.848+0000 7fdf25619200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:31:04.387 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:04.129+0000 7fdf25619200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:31:04.387 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:04.387+0000 7fdf25619200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:31:04.763 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:04.623+0000 7fdf25619200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:31:05.763 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:05 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:05.414+0000 7fdf25619200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:31:06.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:05 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:05.979+0000 7fdf25619200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:31:06.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:06.076+0000 7fdf25619200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:31:06.763 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:06.290+0000 7fdf25619200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:31:06.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: Active manager daemon y restarted 2024-02-07T13:31:06.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: Activating manager daemon y 2024-02-07T13:31:06.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: osdmap e2: 0 total, 0 up, 0 in 2024-02-07T13:31:06.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: mgrmap e6: y(active, starting, since 0.00307866s) 2024-02-07T13:31:06.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:31:06.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:31:06.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:31:06.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:31:06.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:31:06.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: Manager daemon y is now available 2024-02-07T13:31:06.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:06.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:06.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:06.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:06 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:07.616 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:07 smithi094 ceph-mon[25576]: Found migration_current of "None". Setting to last migration. 2024-02-07T13:31:07.616 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:07 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:07.616 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:07 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:31:07.616 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:07 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:31:07.616 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:07 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:07.616 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:07 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:07.617 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:07 smithi094 ceph-mon[25576]: mgrmap e7: y(active, since 1.00556s) 2024-02-07T13:31:07.887 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:31:07.887 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-07T13:31:07.887 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-07T13:31:07.887 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:31:07.887 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 5 is available 2024-02-07T13:31:07.887 INFO:teuthology.orchestra.run.smithi094.stdout:Setting orchestrator backend to cephadm... 2024-02-07T13:31:08.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:08 smithi094 ceph-mon[25576]: [07/Feb/2024:13:31:07] ENGINE Bus STARTING 2024-02-07T13:31:08.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:08 smithi094 ceph-mon[25576]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-07T13:31:08.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:08 smithi094 ceph-mon[25576]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-07T13:31:08.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:08 smithi094 ceph-mon[25576]: [07/Feb/2024:13:31:07] ENGINE Serving on http://172.21.15.94:8765 2024-02-07T13:31:08.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:08 smithi094 ceph-mon[25576]: [07/Feb/2024:13:31:07] ENGINE Serving on https://172.21.15.94:7150 2024-02-07T13:31:08.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:08 smithi094 ceph-mon[25576]: [07/Feb/2024:13:31:07] ENGINE Bus STARTED 2024-02-07T13:31:08.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:08 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:09.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:09 smithi094 ceph-mon[25576]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:09.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:09 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:09.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:09 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:09.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:09 smithi094 ceph-mon[25576]: mgrmap e8: y(active, since 3s) 2024-02-07T13:31:10.340 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-07T13:31:10.340 INFO:teuthology.orchestra.run.smithi094.stdout:Generating ssh key... 2024-02-07T13:31:10.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:10 smithi094 ceph-mon[25576]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: Generating public/private rsa key pair. 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: Your identification has been saved in /tmp/tmpf47ela_0/key. 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: Your public key has been saved in /tmp/tmpf47ela_0/key.pub. 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: The key fingerprint is: 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: SHA256:YUsGRxUM6CLz+YmYogHJAtmxSD7B6fPTYNMw8hlKSDU ceph-edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: The key's randomart image is: 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: +---[RSA 3072]----+ 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: |+++E .o++o. | 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: |+O.*. .o . | 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: |*o* *. = | 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: |o=+*...+ o | 2024-02-07T13:31:11.461 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: |+.+++o S | 2024-02-07T13:31:11.462 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: |o oo. | 2024-02-07T13:31:11.462 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: |. o.o . | 2024-02-07T13:31:11.462 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: |..o . o | 2024-02-07T13:31:11.462 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: |o. | 2024-02-07T13:31:11.462 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:11 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: +----[SHA256]-----+ 2024-02-07T13:31:12.431 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:12 smithi094 ceph-mon[25576]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:12.431 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:12 smithi094 ceph-mon[25576]: Generating ssh key... 2024-02-07T13:31:12.431 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:12 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:12.431 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:12 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:12.986 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGP/WgEmn/PZTZZUYmYibMpQcWd0SZ3zN5ZTLQeDCQPwlpEZrhb9foshla4vnyyR9n51LzvqnvPiqVWCUU86ASePhdTKu+Q+NBjekorjGR1Bk5bB/SbjpKatVsE93DDaXvFRr57rb4s4jIs/Q7J1DyEbU9yfm4RUEOjh+za463fw7npkt4nm9bOJ9e0dRetxWNEM8glg+ZGdPUqEqnZjPcREFZSFDjmfygJ0JvBPHkpcqcOsw3n6VoHppkQ+jkhTsVHTl+Piu5oqO6xuF5NV8Aw4BynYQgckTlTjynTi56sO8sU4Lo8WOXMxm3QcJQEZVQokHYFs/fhOvHKPglchvMtuTX//AWSP03TFkcu7Ebd90vuf1Ui6b7Outw8tPN+64wPtZHOQI4sJGgiqO+zvdjVn80LP3nqAKEWeloLPt23sbbFBQdB8dmFAEFTAA3QnCt7B7UNXLDXRzDRZWYK4Gfl1eiihRJaqAcVjz+39nijp0SG5hI9Utxz3zbuhGwQ/M= ceph-edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:31:12.986 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-07T13:31:12.986 INFO:teuthology.orchestra.run.smithi094.stdout:Adding key to root@localhost authorized_keys... 2024-02-07T13:31:12.987 INFO:teuthology.orchestra.run.smithi094.stdout:Adding host smithi094... 2024-02-07T13:31:13.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:13 smithi094 ceph-mon[25576]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:15.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:14 smithi094 ceph-mon[25576]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:15.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:14 smithi094 ceph-mon[25576]: Deploying cephadm binary to smithi094 2024-02-07T13:31:18.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Added host 'smithi094' with addr '172.21.15.94' 2024-02-07T13:31:18.404 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mon service... 2024-02-07T13:31:18.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:18 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:18.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:18 smithi094 ceph-mon[25576]: Added host smithi094 2024-02-07T13:31:18.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:18 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:20.230 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:20 smithi094 ceph-mon[25576]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:20.230 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:20 smithi094 ceph-mon[25576]: Saving service mon spec with placement count:5 2024-02-07T13:31:20.230 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:20 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:20.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-07T13:31:20.232 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mgr service... 2024-02-07T13:31:21.881 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:21 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:21.881 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:21 smithi094 ceph-mon[25576]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:21.881 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:21 smithi094 ceph-mon[25576]: Saving service mgr spec with placement count:2 2024-02-07T13:31:21.881 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:21 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:21.881 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:21 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:21.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-07T13:31:23.760 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:23 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1894576240' entity='client.admin' 2024-02-07T13:31:24.566 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling the dashboard module... 2024-02-07T13:31:24.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:24 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1969230624' entity='client.admin' 2024-02-07T13:31:24.940 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:24 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:26.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:25 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2775985078' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-07T13:31:26.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:25 smithi094 ceph-mon[25576]: from='mgr.14120 172.21.15.94:0/2408742352' entity='mgr.y' 2024-02-07T13:31:26.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ignoring --setuser ceph since I am not root 2024-02-07T13:31:26.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ignoring --setgroup ceph since I am not root 2024-02-07T13:31:26.379 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:26 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:26.312+0000 7f7fd6bbb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:31:26.707 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:26 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:26.404+0000 7f7fd6bbb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:31:26.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:26 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2775985078' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-07T13:31:26.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:26 smithi094 ceph-mon[25576]: mgrmap e9: y(active, since 19s) 2024-02-07T13:31:27.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:27 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:27.083+0000 7f7fd6bbb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:31:27.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:27 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:27.176+0000 7f7fd6bbb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:31:27.614 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:27 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:27.320+0000 7f7fd6bbb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:31:27.614 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:27 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:27.416+0000 7f7fd6bbb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:31:27.614 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:27 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:27.499+0000 7f7fd6bbb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:31:27.876 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:27 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/270551309' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-07T13:31:27.876 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:27 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:27.765+0000 7f7fd6bbb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:31:27.876 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:27 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:27.876+0000 7f7fd6bbb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:31:28.175 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:27 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:27.963+0000 7f7fd6bbb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:31:28.175 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:28 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:28.058+0000 7f7fd6bbb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:31:28.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:31:28.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-07T13:31:28.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-07T13:31:28.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-07T13:31:28.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-07T13:31:28.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:31:28.346 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-02-07T13:31:28.346 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 9... 2024-02-07T13:31:28.470 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:28 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:28.241+0000 7f7fd6bbb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:31:29.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:29 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:29.236+0000 7f7fd6bbb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:31:29.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:29 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:29.366+0000 7f7fd6bbb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:31:29.984 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:29 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:29.698+0000 7f7fd6bbb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:31:29.984 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:29 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:29.788+0000 7f7fd6bbb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:31:29.985 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:29 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:29.983+0000 7f7fd6bbb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:31:30.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:30.168+0000 7f7fd6bbb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:31:30.710 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:30.452+0000 7f7fd6bbb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:31:30.711 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:30.709+0000 7f7fd6bbb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:31:31.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:30.945+0000 7f7fd6bbb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:31:32.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:31 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:31.738+0000 7f7fd6bbb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:31:32.622 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:32.305+0000 7f7fd6bbb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:31:32.622 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:32.403+0000 7f7fd6bbb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:31:32.622 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:31:32.621+0000 7f7fd6bbb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:31:33.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: Active manager daemon y restarted 2024-02-07T13:31:33.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: Activating manager daemon y 2024-02-07T13:31:33.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: osdmap e3: 0 total, 0 up, 0 in 2024-02-07T13:31:33.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: mgrmap e10: y(active, starting, since 0.00331184s) 2024-02-07T13:31:33.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:31:33.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:31:33.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:31:33.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:31:33.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:31:33.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: Manager daemon y is now available 2024-02-07T13:31:33.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:33.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:33.993 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:33 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:31:33.993 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:33 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:31:33.993 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:33 smithi094 ceph-mon[25576]: mgrmap e11: y(active, since 1.00594s) 2024-02-07T13:31:34.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:31:34.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-07T13:31:34.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-07T13:31:34.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:31:34.522 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 9 is available 2024-02-07T13:31:34.522 INFO:teuthology.orchestra.run.smithi094.stdout:Generating a dashboard self-signed certificate... 2024-02-07T13:31:34.880 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:34 smithi094 ceph-mon[25576]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-07T13:31:34.880 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:34 smithi094 ceph-mon[25576]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-07T13:31:34.880 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:35.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:35 smithi094 ceph-mon[25576]: [07/Feb/2024:13:31:33] ENGINE Bus STARTING 2024-02-07T13:31:35.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:35 smithi094 ceph-mon[25576]: [07/Feb/2024:13:31:34] ENGINE Serving on http://172.21.15.94:8765 2024-02-07T13:31:35.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:35 smithi094 ceph-mon[25576]: [07/Feb/2024:13:31:34] ENGINE Serving on https://172.21.15.94:7150 2024-02-07T13:31:35.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:35 smithi094 ceph-mon[25576]: [07/Feb/2024:13:31:34] ENGINE Bus STARTED 2024-02-07T13:31:35.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:35 smithi094 ceph-mon[25576]: mgrmap e12: y(active, since 2s) 2024-02-07T13:31:35.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:35.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:35.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-07T13:31:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial admin user... 2024-02-07T13:31:36.748 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:36 smithi094 ceph-mon[25576]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:37.471 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$19YEnRttO5Pkn8/n5XmbKOQlK8PxUxY4gEh714infTJcPxZJQaZui", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707312696, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-07T13:31:37.471 INFO:teuthology.orchestra.run.smithi094.stdout:Fetching dashboard port number... 2024-02-07T13:31:38.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:37 smithi094 ceph-mon[25576]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:38.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:37 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:39.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 8443 2024-02-07T13:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T13:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-02-07T13:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T13:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-07T13:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-02-07T13:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-07T13:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph Dashboard is now available at: 2024-02-07T13:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: URL: https://smithi094.front.sepia.ceph.com:8443/ 2024-02-07T13:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: User: admin 2024-02-07T13:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: Password: sw8n0hscn4 2024-02-07T13:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout:Saving cluster configuration to /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config directory 2024-02-07T13:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling autotune for osd_memory_target 2024-02-07T13:31:39.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:39 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3776539050' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-07T13:31:40.570 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:40 smithi094 ceph-mon[25576]: mgrmap e13: y(active, since 6s) 2024-02-07T13:31:41.427 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout:Or, if you are only running a single cluster on this host: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: ceph telemetry on 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout:For more information see: 2024-02-07T13:31:41.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:41.429 INFO:teuthology.orchestra.run.smithi094.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-07T13:31:41.429 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:41.429 INFO:teuthology.orchestra.run.smithi094.stdout:Bootstrap complete. 2024-02-07T13:31:41.473 INFO:tasks.cephadm:Fetching config... 2024-02-07T13:31:41.473 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:31:41.473 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-07T13:31:41.489 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-07T13:31:41.489 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:31:41.490 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-07T13:31:41.545 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-07T13:31:41.546 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:31:41.546 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-07T13:31:41.614 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-07T13:31:41.614 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:31:41.614 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-07T13:31:41.669 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-07T13:31:41.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGP/WgEmn/PZTZZUYmYibMpQcWd0SZ3zN5ZTLQeDCQPwlpEZrhb9foshla4vnyyR9n51LzvqnvPiqVWCUU86ASePhdTKu+Q+NBjekorjGR1Bk5bB/SbjpKatVsE93DDaXvFRr57rb4s4jIs/Q7J1DyEbU9yfm4RUEOjh+za463fw7npkt4nm9bOJ9e0dRetxWNEM8glg+ZGdPUqEqnZjPcREFZSFDjmfygJ0JvBPHkpcqcOsw3n6VoHppkQ+jkhTsVHTl+Piu5oqO6xuF5NV8Aw4BynYQgckTlTjynTi56sO8sU4Lo8WOXMxm3QcJQEZVQokHYFs/fhOvHKPglchvMtuTX//AWSP03TFkcu7Ebd90vuf1Ui6b7Outw8tPN+64wPtZHOQI4sJGgiqO+zvdjVn80LP3nqAKEWeloLPt23sbbFBQdB8dmFAEFTAA3QnCt7B7UNXLDXRzDRZWYK4Gfl1eiihRJaqAcVjz+39nijp0SG5hI9Utxz3zbuhGwQ/M= ceph-edf33ada-c5bc-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-07T13:31:41.747 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGP/WgEmn/PZTZZUYmYibMpQcWd0SZ3zN5ZTLQeDCQPwlpEZrhb9foshla4vnyyR9n51LzvqnvPiqVWCUU86ASePhdTKu+Q+NBjekorjGR1Bk5bB/SbjpKatVsE93DDaXvFRr57rb4s4jIs/Q7J1DyEbU9yfm4RUEOjh+za463fw7npkt4nm9bOJ9e0dRetxWNEM8glg+ZGdPUqEqnZjPcREFZSFDjmfygJ0JvBPHkpcqcOsw3n6VoHppkQ+jkhTsVHTl+Piu5oqO6xuF5NV8Aw4BynYQgckTlTjynTi56sO8sU4Lo8WOXMxm3QcJQEZVQokHYFs/fhOvHKPglchvMtuTX//AWSP03TFkcu7Ebd90vuf1Ui6b7Outw8tPN+64wPtZHOQI4sJGgiqO+zvdjVn80LP3nqAKEWeloLPt23sbbFBQdB8dmFAEFTAA3QnCt7B7UNXLDXRzDRZWYK4Gfl1eiihRJaqAcVjz+39nijp0SG5hI9Utxz3zbuhGwQ/M= ceph-edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:31:41.760 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGP/WgEmn/PZTZZUYmYibMpQcWd0SZ3zN5ZTLQeDCQPwlpEZrhb9foshla4vnyyR9n51LzvqnvPiqVWCUU86ASePhdTKu+Q+NBjekorjGR1Bk5bB/SbjpKatVsE93DDaXvFRr57rb4s4jIs/Q7J1DyEbU9yfm4RUEOjh+za463fw7npkt4nm9bOJ9e0dRetxWNEM8glg+ZGdPUqEqnZjPcREFZSFDjmfygJ0JvBPHkpcqcOsw3n6VoHppkQ+jkhTsVHTl+Piu5oqO6xuF5NV8Aw4BynYQgckTlTjynTi56sO8sU4Lo8WOXMxm3QcJQEZVQokHYFs/fhOvHKPglchvMtuTX//AWSP03TFkcu7Ebd90vuf1Ui6b7Outw8tPN+64wPtZHOQI4sJGgiqO+zvdjVn80LP3nqAKEWeloLPt23sbbFBQdB8dmFAEFTAA3QnCt7B7UNXLDXRzDRZWYK4Gfl1eiihRJaqAcVjz+39nijp0SG5hI9Utxz3zbuhGwQ/M= ceph-edf33ada-c5bc-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-07T13:31:41.807 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGP/WgEmn/PZTZZUYmYibMpQcWd0SZ3zN5ZTLQeDCQPwlpEZrhb9foshla4vnyyR9n51LzvqnvPiqVWCUU86ASePhdTKu+Q+NBjekorjGR1Bk5bB/SbjpKatVsE93DDaXvFRr57rb4s4jIs/Q7J1DyEbU9yfm4RUEOjh+za463fw7npkt4nm9bOJ9e0dRetxWNEM8glg+ZGdPUqEqnZjPcREFZSFDjmfygJ0JvBPHkpcqcOsw3n6VoHppkQ+jkhTsVHTl+Piu5oqO6xuF5NV8Aw4BynYQgckTlTjynTi56sO8sU4Lo8WOXMxm3QcJQEZVQokHYFs/fhOvHKPglchvMtuTX//AWSP03TFkcu7Ebd90vuf1Ui6b7Outw8tPN+64wPtZHOQI4sJGgiqO+zvdjVn80LP3nqAKEWeloLPt23sbbFBQdB8dmFAEFTAA3QnCt7B7UNXLDXRzDRZWYK4Gfl1eiihRJaqAcVjz+39nijp0SG5hI9Utxz3zbuhGwQ/M= ceph-edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:31:41.822 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGP/WgEmn/PZTZZUYmYibMpQcWd0SZ3zN5ZTLQeDCQPwlpEZrhb9foshla4vnyyR9n51LzvqnvPiqVWCUU86ASePhdTKu+Q+NBjekorjGR1Bk5bB/SbjpKatVsE93DDaXvFRr57rb4s4jIs/Q7J1DyEbU9yfm4RUEOjh+za463fw7npkt4nm9bOJ9e0dRetxWNEM8glg+ZGdPUqEqnZjPcREFZSFDjmfygJ0JvBPHkpcqcOsw3n6VoHppkQ+jkhTsVHTl+Piu5oqO6xuF5NV8Aw4BynYQgckTlTjynTi56sO8sU4Lo8WOXMxm3QcJQEZVQokHYFs/fhOvHKPglchvMtuTX//AWSP03TFkcu7Ebd90vuf1Ui6b7Outw8tPN+64wPtZHOQI4sJGgiqO+zvdjVn80LP3nqAKEWeloLPt23sbbFBQdB8dmFAEFTAA3QnCt7B7UNXLDXRzDRZWYK4Gfl1eiihRJaqAcVjz+39nijp0SG5hI9Utxz3zbuhGwQ/M= ceph-edf33ada-c5bc-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-07T13:31:41.867 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGP/WgEmn/PZTZZUYmYibMpQcWd0SZ3zN5ZTLQeDCQPwlpEZrhb9foshla4vnyyR9n51LzvqnvPiqVWCUU86ASePhdTKu+Q+NBjekorjGR1Bk5bB/SbjpKatVsE93DDaXvFRr57rb4s4jIs/Q7J1DyEbU9yfm4RUEOjh+za463fw7npkt4nm9bOJ9e0dRetxWNEM8glg+ZGdPUqEqnZjPcREFZSFDjmfygJ0JvBPHkpcqcOsw3n6VoHppkQ+jkhTsVHTl+Piu5oqO6xuF5NV8Aw4BynYQgckTlTjynTi56sO8sU4Lo8WOXMxm3QcJQEZVQokHYFs/fhOvHKPglchvMtuTX//AWSP03TFkcu7Ebd90vuf1Ui6b7Outw8tPN+64wPtZHOQI4sJGgiqO+zvdjVn80LP3nqAKEWeloLPt23sbbFBQdB8dmFAEFTAA3QnCt7B7UNXLDXRzDRZWYK4Gfl1eiihRJaqAcVjz+39nijp0SG5hI9Utxz3zbuhGwQ/M= ceph-edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:31:41.882 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-07T13:31:41.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:41 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3627610980' entity='client.admin' 2024-02-07T13:31:42.136 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:31:45.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:45.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:45.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:31:45.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:45.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:45.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:45.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:45 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1843671558' entity='client.admin' 2024-02-07T13:31:45.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:45.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:45.550 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-07T13:31:45.550 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-07T13:31:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:31:48.719 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-02-07T13:31:48.719 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:31:48.720 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-02-07T13:31:48.741 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:31:48.741 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:31:48.798 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-02-07T13:31:48.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch host add smithi138 2024-02-07T13:31:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:31:49.246 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:49.246 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:49.246 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:49.246 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:31:49.247 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:31:49.247 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-07T13:31:49.247 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:31:49.247 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:31:49.247 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:31:49.247 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:49.247 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:49.247 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:48 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:51.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:50 smithi094 ceph-mon[25576]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:31:52.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:51 smithi094 ceph-mon[25576]: Deploying cephadm binary to smithi138 2024-02-07T13:31:54.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:53 smithi094 ceph-mon[25576]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:31:55.007 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-02-07T13:31:55.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-07T13:31:55.865 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:31:55.984 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:55 smithi094 ceph-mon[25576]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:31:55.985 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:55.985 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:31:55.985 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:56.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:56 smithi094 ceph-mon[25576]: Added host smithi138 2024-02-07T13:31:56.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:56 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:57.508 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:31:57.508 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-02-07T13:31:58.003 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:57 smithi094 ceph-mon[25576]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:31:58.003 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:57 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:31:58.028 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2024-02-07T13:31:58.028 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:31:58.028 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2024-02-07T13:31:58.047 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:31:58.047 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:31:58.104 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2024-02-07T13:31:58.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch host add smithi190 2024-02-07T13:31:58.334 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:31:58.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:58 smithi094 ceph-mon[25576]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:31:59.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:31:59 smithi094 ceph-mon[25576]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:02.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:01 smithi094 ceph-mon[25576]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:02.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:01 smithi094 ceph-mon[25576]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:02.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:01 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:02.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:01 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:02.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:01 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:02.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:01 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:02.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:01 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:32:02.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:01 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:02.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:01 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:03.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:02 smithi094 ceph-mon[25576]: Deploying cephadm binary to smithi190 2024-02-07T13:32:03.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:02 smithi094 ceph-mon[25576]: Updating smithi138:/etc/ceph/ceph.conf 2024-02-07T13:32:03.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:02 smithi094 ceph-mon[25576]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:03.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:02 smithi094 ceph-mon[25576]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:32:04.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:03 smithi094 ceph-mon[25576]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:04.347 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi190' with addr '172.21.15.190' 2024-02-07T13:32:05.007 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-07T13:32:05.239 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:32:05.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:05 smithi094 ceph-mon[25576]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:32:05.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:05 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:05.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:05 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:05.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:05 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:05.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:05 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:05.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:05 smithi094 ceph-mon[25576]: Added host smithi190 2024-02-07T13:32:05.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:05 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:05.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:05 smithi094 ceph-mon[25576]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:05.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:05 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:06.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:06 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:32:06.997 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2024-02-07T13:32:07.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:07.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:07 smithi094 ceph-mon[25576]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:07.570 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-07T13:32:07.570 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph osd crush tunables default 2024-02-07T13:32:07.803 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:32:08.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:08 smithi094 ceph-mon[25576]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:32:09.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:09 smithi094 ceph-mon[25576]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:10.437 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-02-07T13:32:10.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:10 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1079285903' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-07T13:32:10.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:10.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:10.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:10.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:10.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:32:10.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:10.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:11.038 INFO:tasks.cephadm:Adding mon.a on smithi094 2024-02-07T13:32:11.038 INFO:tasks.cephadm:Adding mon.b on smithi138 2024-02-07T13:32:11.039 INFO:tasks.cephadm:Adding mon.c on smithi190 2024-02-07T13:32:11.039 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch apply mon '3;smithi094:172.21.15.94=a;smithi138:172.21.15.138=b;smithi190:172.21.15.190=c' 2024-02-07T13:32:11.265 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:11.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: Updating smithi190:/etc/ceph/ceph.conf 2024-02-07T13:32:11.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:11.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:32:11.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1079285903' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-07T13:32:11.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:11.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:32:11.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:11.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:11.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:11.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:12.863 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2024-02-07T13:32:13.449 DEBUG:teuthology.orchestra.run.smithi138:mon.b> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.b.service 2024-02-07T13:32:13.451 DEBUG:teuthology.orchestra.run.smithi190:mon.c> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.c.service 2024-02-07T13:32:13.454 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-07T13:32:13.454 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-07T13:32:14.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:13 smithi094 ceph-mon[25576]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:14.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:13 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:14.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:13 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:14.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:13 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:14.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:13 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:14.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:13 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:14.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:13 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:14.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:13 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:14.230 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:15.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:14 smithi094 ceph-mon[25576]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi138:172.21.15.138=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:15.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:14 smithi094 ceph-mon[25576]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi138:172.21.15.138=b;smithi190:172.21.15.190=c;count:3 2024-02-07T13:32:15.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:14 smithi094 ceph-mon[25576]: Deploying daemon mon.c on smithi190 2024-02-07T13:32:16.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:15 smithi094 ceph-mon[25576]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:17.510 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:32:17.510 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"edf33ada-c5bc-11ee-95b6-87774f69a715","modified":"2024-02-07T13:30:30.384504Z","created":"2024-02-07T13:30:30.384504Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-07T13:32:17.511 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-02-07T13:32:17.670 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).mds e1 new map 2024-02-07T13:32:17.670 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-07T13:32:17.670 INFO:journalctl@ceph.mon.c.smithi190.stdout: e1 2024-02-07T13:32:17.670 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout: legacy client fscid: -1 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout: No filesystems configured 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-07T13:32:17.671 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-07T13:32:18.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-07T13:32:18.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:32:18.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:32:18.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:32:18.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:32:18.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Deploying cephadm binary to smithi190 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Updating smithi138:/etc/ceph/ceph.conf 2024-02-07T13:32:18.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Added host smithi190 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1079285903' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Updating smithi190:/etc/ceph/ceph.conf 2024-02-07T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1079285903' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:18.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi138:172.21.15.138=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:18.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi138:172.21.15.138=b;smithi190:172.21.15.190=c;count:3 2024-02-07T13:32:18.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: Deploying daemon mon.c on smithi190 2024-02-07T13:32:18.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:18.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 ceph-mon[30713]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-07T13:32:18.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 bash[30664]: 6bd712409c6ba66ae92b8972ba24261f1e288f5df0ae3fc65e7442467c0fc3e0 2024-02-07T13:32:18.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:17 smithi190 systemd[1]: Started Ceph mon.c for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:32:19.327 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-07T13:32:19.328 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-07T13:32:19.574 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:32:23.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: mon.a calling monitor election 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: mon.c calling monitor election 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: monmap e2: 2 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: fsmap 2024-02-07T13:32:23.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:23.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: mgrmap e13: y(active, since 50s) 2024-02-07T13:32:23.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: overall HEALTH_OK 2024-02-07T13:32:23.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:23.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:23.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:32:23.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: mon.a calling monitor election 2024-02-07T13:32:23.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:23.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: mon.c calling monitor election 2024-02-07T13:32:23.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: monmap e2: 2 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: fsmap 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: mgrmap e13: y(active, since 50s) 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: overall HEALTH_OK 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:23.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:23 smithi094 ceph-mon[25576]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:23 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:23 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:23 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:23 smithi094 ceph-mon[25576]: Deploying daemon mon.b on smithi138 2024-02-07T13:32:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:23 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:23 smithi190 ceph-mon[30713]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:23 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:23 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:23 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:23 smithi190 ceph-mon[30713]: Deploying daemon mon.b on smithi138 2024-02-07T13:32:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:23 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:24.227 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:32:24.228 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"edf33ada-c5bc-11ee-95b6-87774f69a715","modified":"2024-02-07T13:32:17.675993Z","created":"2024-02-07T13:30:30.384504Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-07T13:32:24.231 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-02-07T13:32:24.802 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:24 smithi190 ceph-mon[30713]: from='client.? 172.21.15.190:0/805819534' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:32:25.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:24 smithi094 ceph-mon[25576]: from='client.? 172.21.15.190:0/805819534' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:32:25.829 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-07T13:32:25.830 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-07T13:32:26.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:25 smithi190 ceph-mon[30713]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:26.066 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:32:26.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:25 smithi094 ceph-mon[25576]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:26.751 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:26 smithi138 ceph-mon[30535]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-07T13:32:26.751 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:26 smithi138 bash[30497]: 55ff65143798347d60a14622d12a6eb029bba3723072585c4dbcc28742ce49c5 2024-02-07T13:32:27.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:26 smithi138 systemd[1]: Started Ceph mon.b for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:32:32.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: mon.a calling monitor election 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: mon.c calling monitor election 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: fsmap 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: mgrmap e13: y(active, since 59s) 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: mon.b (rank 2) addr [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] is down (out of quorum) 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:32.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:32.266 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:32.266 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:32.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:32.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:32:32.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: mon.a calling monitor election 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: mon.c calling monitor election 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: fsmap 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:32.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: mgrmap e13: y(active, since 59s) 2024-02-07T13:32:32.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-07T13:32:32.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-07T13:32:32.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T13:32:32.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: mon.b (rank 2) addr [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] is down (out of quorum) 2024-02-07T13:32:32.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:32.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:32.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:32.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:32.325 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-02-07T13:32:32.325 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":3,"fsid":"edf33ada-c5bc-11ee-95b6-87774f69a715","modified":"2024-02-07T13:32:26.734142Z","created":"2024-02-07T13:30:30.384504Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-07T13:32:32.328 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 3 2024-02-07T13:32:32.914 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:32 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:32.914 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:32 smithi190 ceph-mon[30713]: from='client.? 172.21.15.190:0/548283724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:32:32.914 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:32 smithi190 ceph-mon[30713]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:32.914 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:32 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:32.914 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:32 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:32.946 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-07T13:32:32.946 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph config generate-minimal-conf 2024-02-07T13:32:33.192 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:32:33.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:33.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:32 smithi094 ceph-mon[25576]: from='client.? 172.21.15.190:0/548283724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:32:33.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:32 smithi094 ceph-mon[25576]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:33.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:33.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:34.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:34.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:32:34.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: mon.a calling monitor election 2024-02-07T13:32:34.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: mon.c calling monitor election 2024-02-07T13:32:34.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:32:34.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: fsmap 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: mgrmap e13: y(active, since 59s) 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: mon.b (rank 2) addr [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] is down (out of quorum) 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:34.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='client.? 172.21.15.190:0/548283724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:32:34.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:34.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:34.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:35.004 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: mon.b calling monitor election 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: Updating smithi138:/etc/ceph/ceph.conf 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: Updating smithi190:/etc/ceph/ceph.conf 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: mon.b calling monitor election 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: mon.c calling monitor election 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: mon.a calling monitor election 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: fsmap 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: mgrmap e13: y(active, since 61s) 2024-02-07T13:32:35.005 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: Cluster is now healthy 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: overall HEALTH_OK 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:35.007 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:32:35.007 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:35.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: mon.b calling monitor election 2024-02-07T13:32:35.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-07T13:32:35.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: Updating smithi138:/etc/ceph/ceph.conf 2024-02-07T13:32:35.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: Updating smithi190:/etc/ceph/ceph.conf 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: mon.b calling monitor election 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: mon.c calling monitor election 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: mon.a calling monitor election 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: fsmap 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: mgrmap e13: y(active, since 61s) 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: Cluster is now healthy 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: overall HEALTH_OK 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:32:35.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:# minimal ceph.conf for edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:32:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-02-07T13:32:35.077 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:32:35.077 INFO:teuthology.orchestra.run.smithi094.stdout: mon_host = [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-02-07T13:32:35.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: mon.b calling monitor election 2024-02-07T13:32:35.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-07T13:32:35.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: Updating smithi138:/etc/ceph/ceph.conf 2024-02-07T13:32:35.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: Updating smithi190:/etc/ceph/ceph.conf 2024-02-07T13:32:35.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: mon.b calling monitor election 2024-02-07T13:32:35.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: mon.c calling monitor election 2024-02-07T13:32:35.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: mon.a calling monitor election 2024-02-07T13:32:35.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-07T13:32:35.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: fsmap 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: mgrmap e13: y(active, since 61s) 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: Cluster is now healthy 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: overall HEALTH_OK 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.097 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:35.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:35.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:32:35.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:35.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:35 smithi094 ceph-mon[25576]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:35.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:35 smithi094 ceph-mon[25576]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:35.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:35 smithi094 ceph-mon[25576]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:35.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:35 smithi094 ceph-mon[25576]: Reconfiguring mon.a (unknown last config time)... 2024-02-07T13:32:35.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:35 smithi094 ceph-mon[25576]: Reconfiguring daemon mon.a on smithi094 2024-02-07T13:32:35.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:35 smithi094 ceph-mon[25576]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:35.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:35.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:35 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/216472726' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:35 smithi190 ceph-mon[30713]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:35 smithi190 ceph-mon[30713]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:35 smithi190 ceph-mon[30713]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:35 smithi190 ceph-mon[30713]: Reconfiguring mon.a (unknown last config time)... 2024-02-07T13:32:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:35 smithi190 ceph-mon[30713]: Reconfiguring daemon mon.a on smithi094 2024-02-07T13:32:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:35 smithi190 ceph-mon[30713]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:36.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:35 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:36.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:35 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/216472726' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:35 smithi138 ceph-mon[30535]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:35 smithi138 ceph-mon[30535]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:35 smithi138 ceph-mon[30535]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:32:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:35 smithi138 ceph-mon[30535]: Reconfiguring mon.a (unknown last config time)... 2024-02-07T13:32:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:35 smithi138 ceph-mon[30535]: Reconfiguring daemon mon.a on smithi094 2024-02-07T13:32:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:35 smithi138 ceph-mon[30535]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:35 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:32:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:35 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/216472726' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:36.349 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-07T13:32:36.349 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:32:36.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-02-07T13:32:36.378 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:32:36.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:32:36.446 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:32:36.446 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-02-07T13:32:36.475 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:32:36.475 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:32:36.544 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:32:36.545 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2024-02-07T13:32:36.579 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:32:36.579 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:32:36.650 INFO:tasks.cephadm:Adding mgr.y on smithi094 2024-02-07T13:32:36.650 INFO:tasks.cephadm:Adding mgr.x on smithi138 2024-02-07T13:32:36.650 INFO:tasks.cephadm:Adding mgr.z on smithi190 2024-02-07T13:32:36.650 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch apply mgr '3;smithi094=y;smithi138=x;smithi190=z' 2024-02-07T13:32:36.923 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:32:37.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:37 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:37.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:37 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:37.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:37 smithi138 ceph-mon[30535]: Reconfiguring mon.b (monmap changed)... 2024-02-07T13:32:37.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:37 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:37.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:37 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:32:37.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:37 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:37.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:37 smithi138 ceph-mon[30535]: Reconfiguring daemon mon.b on smithi138 2024-02-07T13:32:37.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:37 smithi138 ceph-mon[30535]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:37.362 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:37 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:37.363 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:37 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:37.363 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:37 smithi190 ceph-mon[30713]: Reconfiguring mon.b (monmap changed)... 2024-02-07T13:32:37.363 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:37 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:37.363 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:37 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:32:37.363 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:37 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:37.363 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:37 smithi190 ceph-mon[30713]: Reconfiguring daemon mon.b on smithi138 2024-02-07T13:32:37.363 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:37 smithi190 ceph-mon[30713]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:37.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:37 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:37.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:37 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:37.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:37 smithi094 ceph-mon[25576]: Reconfiguring mon.b (monmap changed)... 2024-02-07T13:32:37.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:37 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:37.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:37 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:32:37.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:37 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:37.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:37 smithi094 ceph-mon[25576]: Reconfiguring daemon mon.b on smithi138 2024-02-07T13:32:37.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:37 smithi094 ceph-mon[25576]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:38.687 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mgr update... 2024-02-07T13:32:38.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:38 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:38.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:38 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:38.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:38 smithi190 ceph-mon[30713]: Reconfiguring mon.c (monmap changed)... 2024-02-07T13:32:38.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:38 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:38.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:38 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:32:38.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:38 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:38.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:38 smithi190 ceph-mon[30713]: Reconfiguring daemon mon.c on smithi190 2024-02-07T13:32:39.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:38 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:38 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:38 smithi094 ceph-mon[25576]: Reconfiguring mon.c (monmap changed)... 2024-02-07T13:32:39.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:38 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:39.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:38 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:32:39.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:38 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:39.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:38 smithi094 ceph-mon[25576]: Reconfiguring daemon mon.c on smithi190 2024-02-07T13:32:39.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:38 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:38 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:38 smithi138 ceph-mon[30535]: Reconfiguring mon.c (monmap changed)... 2024-02-07T13:32:39.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:38 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:32:39.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:38 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:32:39.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:38 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:39.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:38 smithi138 ceph-mon[30535]: Reconfiguring daemon mon.c on smithi190 2024-02-07T13:32:39.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:39 smithi138 ceph-mon[30535]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:39.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:39 smithi138 ceph-mon[30535]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=y;smithi138=x;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:39.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:39 smithi138 ceph-mon[30535]: Saving service mgr spec with placement smithi094=y;smithi138=x;smithi190=z;count:3 2024-02-07T13:32:39.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:39 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:39 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:39 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:39 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:39.952 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:39 smithi190 ceph-mon[30713]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:39.952 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:39 smithi190 ceph-mon[30713]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=y;smithi138=x;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:39.952 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:39 smithi190 ceph-mon[30713]: Saving service mgr spec with placement smithi094=y;smithi138=x;smithi190=z;count:3 2024-02-07T13:32:39.953 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:39 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.953 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:39 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.953 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:39 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:39.953 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:39 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:39.977 DEBUG:teuthology.orchestra.run.smithi138:mgr.x> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.x.service 2024-02-07T13:32:39.979 DEBUG:teuthology.orchestra.run.smithi190:mgr.z> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.z.service 2024-02-07T13:32:39.981 INFO:tasks.cephadm:Deploying OSDs... 2024-02-07T13:32:39.981 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:32:39.982 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:32:39.998 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:32:39.998 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/nvme1n1 2024-02-07T13:32:40.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:39 smithi094 ceph-mon[25576]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:40.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:39 smithi094 ceph-mon[25576]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=y;smithi138=x;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:40.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:39 smithi094 ceph-mon[25576]: Saving service mgr spec with placement smithi094=y;smithi138=x;smithi190=z;count:3 2024-02-07T13:32:40.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:39 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:40.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:39 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:40.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:39 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:40.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:39 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/nvme1n1 2024-02-07T13:32:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:40.028 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-07T13:32:40.028 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:40.028 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:40.028 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-07 13:31:45.246697252 +0000 2024-02-07T13:32:40.028 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-07 13:29:29.211299801 +0000 2024-02-07T13:32:40.028 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-07 13:29:29.211299801 +0000 2024-02-07T13:32:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-07 13:29:29.200300000 +0000 2024-02-07T13:32:40.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-07T13:32:40.093 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-07T13:32:40.094 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-07T13:32:40.094 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000134761 s, 3.8 MB/s 2024-02-07T13:32:40.095 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-07T13:32:40.150 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/nvme2n1 2024-02-07T13:32:40.205 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/nvme2n1 2024-02-07T13:32:40.206 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:40.206 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-07T13:32:40.206 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:40.206 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:40.206 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-07 13:31:45.254697108 +0000 2024-02-07T13:32:40.206 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-07 13:29:29.356297181 +0000 2024-02-07T13:32:40.206 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-07 13:29:29.356297181 +0000 2024-02-07T13:32:40.206 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-07 13:29:29.346297361 +0000 2024-02-07T13:32:40.206 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-07T13:32:40.271 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-07T13:32:40.271 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-07T13:32:40.271 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000124115 s, 4.1 MB/s 2024-02-07T13:32:40.272 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-07T13:32:40.328 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/nvme3n1 2024-02-07T13:32:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/nvme3n1 2024-02-07T13:32:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:40.383 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-07T13:32:40.383 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:40.383 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:40.383 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-07 13:31:45.262696964 +0000 2024-02-07T13:32:40.384 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-07 13:29:29.492294722 +0000 2024-02-07T13:32:40.384 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-07 13:29:29.492294722 +0000 2024-02-07T13:32:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-07 13:29:29.489294776 +0000 2024-02-07T13:32:40.384 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-07T13:32:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-07T13:32:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-07T13:32:40.449 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000130806 s, 3.9 MB/s 2024-02-07T13:32:40.450 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-07T13:32:40.506 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/nvme4n1 2024-02-07T13:32:40.560 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/nvme4n1 2024-02-07T13:32:40.561 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:40.561 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-07T13:32:40.561 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:40.561 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:40.561 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-07 13:31:45.271696802 +0000 2024-02-07T13:32:40.561 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-07 13:29:29.642292011 +0000 2024-02-07T13:32:40.561 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-07 13:29:29.642292011 +0000 2024-02-07T13:32:40.561 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-07 13:29:29.633292174 +0000 2024-02-07T13:32:40.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-07T13:32:40.625 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-07T13:32:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-07T13:32:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.00012813 s, 4.0 MB/s 2024-02-07T13:32:40.627 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-07T13:32:40.682 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:32:40.682 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:32:40.701 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:32:40.701 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/nvme1n1 2024-02-07T13:32:40.758 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/nvme1n1 2024-02-07T13:32:40.758 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:40.758 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-07T13:32:40.759 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:40.759 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:40.759 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-02-07 13:31:59.894887076 +0000 2024-02-07T13:32:40.759 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-02-07 13:29:30.820163668 +0000 2024-02-07T13:32:40.759 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-02-07 13:29:30.820163668 +0000 2024-02-07T13:32:40.759 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-02-07 13:29:30.815164508 +0000 2024-02-07T13:32:40.759 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-07T13:32:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:40 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:40 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:40 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:40 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:32:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:40 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:32:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:40 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:32:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:40 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-02-07T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-02-07T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000244001 s, 2.1 MB/s 2024-02-07T13:32:40.829 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-07T13:32:40.871 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:40 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:40.872 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:40 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:40.872 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:40 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:40.872 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:40 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:32:40.872 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:40 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:32:40.872 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:40 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:32:40.872 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:40 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:40.888 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/nvme2n1 2024-02-07T13:32:40.944 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/nvme2n1 2024-02-07T13:32:40.944 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:40.944 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-07T13:32:40.945 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:40.945 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:40.945 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-02-07 13:31:59.904886863 +0000 2024-02-07T13:32:40.945 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-02-07 13:29:30.972138126 +0000 2024-02-07T13:32:40.945 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-02-07 13:29:30.972138126 +0000 2024-02-07T13:32:40.945 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-02-07 13:29:30.961139975 +0000 2024-02-07T13:32:40.945 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-07T13:32:41.011 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:40 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:41.011 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:40 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:41.011 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:40 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:41.011 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:40 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:32:41.011 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:40 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:32:41.011 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:40 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:32:41.011 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:40 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:41.013 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-02-07T13:32:41.013 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-02-07T13:32:41.013 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000203198 s, 2.5 MB/s 2024-02-07T13:32:41.014 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-07T13:32:41.072 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/nvme3n1 2024-02-07T13:32:41.128 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/nvme3n1 2024-02-07T13:32:41.128 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:41.128 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-07T13:32:41.129 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:41.129 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:41.129 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-02-07 13:31:59.914886649 +0000 2024-02-07T13:32:41.129 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-02-07 13:29:31.115114097 +0000 2024-02-07T13:32:41.129 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-02-07 13:29:31.115114097 +0000 2024-02-07T13:32:41.129 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-02-07 13:29:31.105115777 +0000 2024-02-07T13:32:41.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-07T13:32:41.196 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-02-07T13:32:41.197 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-02-07T13:32:41.197 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000233826 s, 2.2 MB/s 2024-02-07T13:32:41.198 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-07T13:32:41.255 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/nvme4n1 2024-02-07T13:32:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/nvme4n1 2024-02-07T13:32:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:41.312 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-07T13:32:41.312 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:41.312 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:41.312 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-02-07 13:31:59.924886435 +0000 2024-02-07T13:32:41.312 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-02-07 13:29:31.259089899 +0000 2024-02-07T13:32:41.312 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-02-07 13:29:31.259089899 +0000 2024-02-07T13:32:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-02-07 13:29:31.248091748 +0000 2024-02-07T13:32:41.312 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-07T13:32:41.379 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-02-07T13:32:41.379 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-02-07T13:32:41.379 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000215634 s, 2.4 MB/s 2024-02-07T13:32:41.380 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-07T13:32:41.436 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:32:41.436 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:32:41.453 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:32:41.453 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/nvme1n1 2024-02-07T13:32:41.509 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/nvme1n1 2024-02-07T13:32:41.509 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:41.509 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 764 Links: 1 Device type: 103,2 2024-02-07T13:32:41.509 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:41.509 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:41.509 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-02-07 13:32:08.851367793 +0000 2024-02-07T13:32:41.509 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-02-07 13:29:32.462878907 +0000 2024-02-07T13:32:41.509 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-02-07 13:29:32.462878907 +0000 2024-02-07T13:32:41.509 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-02-07 13:29:32.453879167 +0000 2024-02-07T13:32:41.509 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-07T13:32:41.576 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-02-07T13:32:41.576 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-02-07T13:32:41.576 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000217328 s, 2.4 MB/s 2024-02-07T13:32:41.578 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-07T13:32:41.634 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/nvme2n1 2024-02-07T13:32:41.690 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/nvme2n1 2024-02-07T13:32:41.690 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:41.690 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-07T13:32:41.690 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:41.690 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:41.690 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-02-07 13:32:08.861367506 +0000 2024-02-07T13:32:41.691 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-02-07 13:29:32.608874681 +0000 2024-02-07T13:32:41.691 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-02-07 13:29:32.608874681 +0000 2024-02-07T13:32:41.691 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-02-07 13:29:32.597874999 +0000 2024-02-07T13:32:41.691 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-07T13:32:41.757 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-02-07T13:32:41.758 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-02-07T13:32:41.758 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000553002 s, 926 kB/s 2024-02-07T13:32:41.759 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-07T13:32:41.815 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/nvme3n1 2024-02-07T13:32:41.876 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/nvme3n1 2024-02-07T13:32:41.877 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:41.877 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-07T13:32:41.877 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:41.877 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:41.877 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-02-07 13:32:08.871367218 +0000 2024-02-07T13:32:41.877 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-02-07 13:29:32.751870542 +0000 2024-02-07T13:32:41.877 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-02-07 13:29:32.751870542 +0000 2024-02-07T13:32:41.877 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-02-07 13:29:32.740870860 +0000 2024-02-07T13:32:41.877 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-07T13:32:41.941 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:41 smithi190 ceph-mon[30713]: Deploying daemon mgr.z on smithi190 2024-02-07T13:32:41.941 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:41 smithi190 ceph-mon[30713]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:41.993 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-02-07T13:32:41.993 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-02-07T13:32:41.993 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000192108 s, 2.7 MB/s 2024-02-07T13:32:41.994 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-07T13:32:42.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:41 smithi094 ceph-mon[25576]: Deploying daemon mgr.z on smithi190 2024-02-07T13:32:42.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:41 smithi094 ceph-mon[25576]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:42.050 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/nvme4n1 2024-02-07T13:32:42.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:41 smithi138 ceph-mon[30535]: Deploying daemon mgr.z on smithi190 2024-02-07T13:32:42.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:41 smithi138 ceph-mon[30535]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:42.105 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/nvme4n1 2024-02-07T13:32:42.106 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:32:42.106 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-07T13:32:42.106 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:32:42.106 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:32:42.106 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-02-07 13:32:08.881366931 +0000 2024-02-07T13:32:42.106 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-02-07 13:29:32.894866403 +0000 2024-02-07T13:32:42.106 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-02-07 13:29:32.894866403 +0000 2024-02-07T13:32:42.106 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-02-07 13:29:32.883866721 +0000 2024-02-07T13:32:42.106 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-07T13:32:42.181 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-02-07T13:32:42.181 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-02-07T13:32:42.181 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000208926 s, 2.5 MB/s 2024-02-07T13:32:42.182 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-07T13:32:42.238 INFO:tasks.cephadm:Deploying osd.0 on smithi094 with /dev/nvme4n1... 2024-02-07T13:32:42.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-07T13:32:42.478 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:32:42.530 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:42 smithi190 podman[32775]: 2024-02-07 13:32:42.420428291 +0000 UTC m=+0.119377126 container create 4df78f4d0932eaffeed1fca88eaeffc93740d0b661cad497ff6f9b380a657c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, io.buildah.version=1.33.2) 2024-02-07T13:32:43.030 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:42 smithi190 podman[32775]: 2024-02-07 13:32:42.639298655 +0000 UTC m=+0.338247522 container init 4df78f4d0932eaffeed1fca88eaeffc93740d0b661cad497ff6f9b380a657c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-02-07T13:32:43.030 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:42 smithi190 podman[32775]: 2024-02-07 13:32:42.645030429 +0000 UTC m=+0.343979273 container start 4df78f4d0932eaffeed1fca88eaeffc93740d0b661cad497ff6f9b380a657c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:32:43.030 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:42 smithi190 ceph-mgr[32814]: -- 172.21.15.190:0/2003848358 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556b9b94f520 con 0x556b9b924800 2024-02-07T13:32:43.031 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:42 smithi190 bash[32775]: 4df78f4d0932eaffeed1fca88eaeffc93740d0b661cad497ff6f9b380a657c40 2024-02-07T13:32:43.031 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:42 smithi190 systemd[1]: Started Ceph mgr.z for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:32:43.530 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:43.187+0000 7f79ed51d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:32:43.530 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:43.278+0000 7f79ed51d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:32:43.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:43 smithi094 ceph-mon[25576]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:43.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:43 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:43.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:43 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:43.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:43 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:43.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:43 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:32:43.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:43 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:32:43.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:43 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:32:43.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:43 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:43.824 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:43 smithi138 ceph-mon[30535]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:43.825 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:43 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:43.825 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:43 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:43.825 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:43 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:43.825 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:43 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:32:43.825 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:43 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:32:43.825 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:43 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:32:43.825 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:43 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:44.000 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:43.909+0000 7f79ed51d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:32:44.000 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:44 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:44.000+0000 7f79ed51d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:32:44.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-mon[30713]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:44.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:44.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:44.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:44.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:32:44.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:32:44.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:32:44.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:43 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:44.280 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:44 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:44.141+0000 7f79ed51d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:32:44.280 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:44 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:44.234+0000 7f79ed51d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:32:44.365 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/nvme4n1 2024-02-07T13:32:44.365 INFO:teuthology.orchestra.run.smithi094.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-07T13:32:44.365 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-07T13:32:44.365 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-02-07T13:32:44.365 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-02-07T13:32:44.365 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00978128 s, 1.1 GB/s 2024-02-07T13:32:44.365 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-02-07T13:32:44.574 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:44 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:44.316+0000 7f79ed51d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:32:44.574 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:44 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:44.574+0000 7f79ed51d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:32:44.834 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:44 smithi190 ceph-mon[30713]: Deploying daemon mgr.x on smithi138 2024-02-07T13:32:44.834 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:44 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:44.681+0000 7f79ed51d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:32:44.834 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:44 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:44.766+0000 7f79ed51d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:32:44.934 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:44 smithi094 ceph-mon[25576]: Deploying daemon mgr.x on smithi138 2024-02-07T13:32:44.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi094:/dev/nvme4n1 2024-02-07T13:32:45.007 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:44 smithi138 ceph-mon[30535]: Deploying daemon mgr.x on smithi138 2024-02-07T13:32:45.008 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:44 smithi138 systemd[1]: Starting Ceph mgr.x for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:32:45.195 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:32:45.239 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:44 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:44.861+0000 7f79ed51d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:32:45.239 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:45 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:45.044+0000 7f79ed51d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:32:45.317 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:45 smithi138 podman[31975]: 2024-02-07 13:32:44.910305225 +0000 UTC m=+0.017510103 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:32:45.317 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:45 smithi138 podman[31975]: 2024-02-07 13:32:45.015061938 +0000 UTC m=+0.122266810 container create dddb105e80c14d15a74676f4c52b1592ccdc753a589a61a9debce5358e663662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T13:32:45.317 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:45 smithi138 podman[31975]: 2024-02-07 13:32:45.225924835 +0000 UTC m=+0.333129714 container init dddb105e80c14d15a74676f4c52b1592ccdc753a589a61a9debce5358e663662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-07T13:32:45.317 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:45 smithi138 podman[31975]: 2024-02-07 13:32:45.231515712 +0000 UTC m=+0.338720589 container start dddb105e80c14d15a74676f4c52b1592ccdc753a589a61a9debce5358e663662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-07T13:32:45.317 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:45 smithi138 ceph-mgr[32014]: -- 172.21.15.138:0/412461849 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f23c187520 con 0x55f23c15c800 2024-02-07T13:32:45.317 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:45 smithi138 bash[31975]: dddb105e80c14d15a74676f4c52b1592ccdc753a589a61a9debce5358e663662 2024-02-07T13:32:45.592 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:45 smithi138 systemd[1]: Started Ceph mgr.x for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:32:46.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:45 smithi094 ceph-mon[25576]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:46.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.006 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.007 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.007 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.007 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:45 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:46.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:45 smithi190 ceph-mon[30713]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:46.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:45 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:45 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:45 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:45 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:45 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:46.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:45 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:45.829+0000 7fee07ebd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:32:46.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:45 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:45.927+0000 7fee07ebd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:32:46.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:45 smithi138 ceph-mon[30535]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:46.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:45 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:45 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:45 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:45 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:46.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:45 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:46.398 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:46 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:46.083+0000 7f79ed51d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:32:46.399 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:46 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:46.223+0000 7f79ed51d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:32:46.657 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:46 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:46.567+0000 7f79ed51d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:32:46.658 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:46 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:46.657+0000 7f79ed51d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:32:46.902 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:46 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:46.646+0000 7fee07ebd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:32:46.903 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:46 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:46.739+0000 7fee07ebd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:32:47.030 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:46 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:46.861+0000 7f79ed51d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:32:47.328 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:47.048+0000 7f79ed51d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:32:47.342 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:46 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:46.902+0000 7fee07ebd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:32:47.343 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:46.999+0000 7fee07ebd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:32:47.343 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:47.084+0000 7fee07ebd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:32:47.589 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:47.328+0000 7f79ed51d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:32:47.589 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:47.588+0000 7f79ed51d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:32:47.641 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:47.355+0000 7fee07ebd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:32:47.641 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:47.463+0000 7fee07ebd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:32:47.642 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:47.548+0000 7fee07ebd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:32:47.642 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:47.641+0000 7fee07ebd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:32:47.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:47.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:47.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:47.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:47.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:47.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:47.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:47.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:47.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:32:47.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:32:47.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:47.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:32:47.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:32:47.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:47 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:48.030 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:47.821+0000 7f79ed51d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:32:48.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:48.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:32:48.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:47 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:48.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:47.818+0000 7fee07ebd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:32:48.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:48.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:48.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:48.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:48.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:32:48.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:32:48.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:48.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:32:48.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:32:48.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:47 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:48.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:48 smithi094 ceph-mon[25576]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:48.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:48 smithi094 ceph-mon[25576]: Reconfiguring mgr.y (unknown last config time)... 2024-02-07T13:32:48.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:48 smithi094 ceph-mon[25576]: Reconfiguring daemon mgr.y on smithi094 2024-02-07T13:32:48.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:48 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:48 smithi190 ceph-mon[30713]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:49.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:48 smithi190 ceph-mon[30713]: Reconfiguring mgr.y (unknown last config time)... 2024-02-07T13:32:49.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:48 smithi190 ceph-mon[30713]: Reconfiguring daemon mgr.y on smithi094 2024-02-07T13:32:49.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:48 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.030 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:48 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:48.611+0000 7f79ed51d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:32:49.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:48 smithi138 ceph-mon[30535]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:32:49.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:48 smithi138 ceph-mon[30535]: Reconfiguring mgr.y (unknown last config time)... 2024-02-07T13:32:49.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:48 smithi138 ceph-mon[30535]: Reconfiguring daemon mgr.y on smithi094 2024-02-07T13:32:49.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:48 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:48 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:48.784+0000 7fee07ebd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:32:49.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:48 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:48.914+0000 7fee07ebd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:32:49.494 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:49.181+0000 7f79ed51d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:32:49.494 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:49.279+0000 7f79ed51d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:32:49.494 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:32:49.494+0000 7f79ed51d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:32:49.530 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:49.248+0000 7fee07ebd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:32:49.530 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:49.338+0000 7fee07ebd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:32:49.530 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:49.529+0000 7fee07ebd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:32:49.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:49.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: Standby manager daemon z started 2024-02-07T13:32:49.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:32:49.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:32:49.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:32:49.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:32:49.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:49.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:49.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:49.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:49 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.818 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: Standby manager daemon z started 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:49 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:49.712+0000 7fee07ebd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: Standby manager daemon z started 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.190:0/914463265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:32:49.844 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:49.844 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:32:49.844 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:32:50.259 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:49 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:49.996+0000 7fee07ebd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:32:50.259 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:50 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:50.259+0000 7fee07ebd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:32:50.592 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:50 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:50.490+0000 7fee07ebd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:32:51.593 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:51.282+0000 7fee07ebd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:32:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-mon[30535]: mgrmap e14: y(active, since 77s), standbys: z 2024-02-07T13:32:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:32:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-mon[30535]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/2841220515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a54e30c-a30a-4824-be90-1326fb7d73fa"}]: dispatch 2024-02-07T13:32:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/2841220515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a54e30c-a30a-4824-be90-1326fb7d73fa"}]': finished 2024-02-07T13:32:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-mon[30535]: osdmap e5: 1 total, 0 up, 1 in 2024-02-07T13:32:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:32:51.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:51 smithi094 ceph-mon[25576]: mgrmap e14: y(active, since 77s), standbys: z 2024-02-07T13:32:51.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:51 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:32:51.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:51 smithi094 ceph-mon[25576]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:51.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:51 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2841220515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a54e30c-a30a-4824-be90-1326fb7d73fa"}]: dispatch 2024-02-07T13:32:51.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:51 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2841220515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a54e30c-a30a-4824-be90-1326fb7d73fa"}]': finished 2024-02-07T13:32:51.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:51 smithi094 ceph-mon[25576]: osdmap e5: 1 total, 0 up, 1 in 2024-02-07T13:32:51.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:51 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:32:51.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:51 smithi190 ceph-mon[30713]: mgrmap e14: y(active, since 77s), standbys: z 2024-02-07T13:32:51.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:51 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:32:51.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:51 smithi190 ceph-mon[30713]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:51.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:51 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/2841220515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a54e30c-a30a-4824-be90-1326fb7d73fa"}]: dispatch 2024-02-07T13:32:51.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:51 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/2841220515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a54e30c-a30a-4824-be90-1326fb7d73fa"}]': finished 2024-02-07T13:32:51.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:51 smithi190 ceph-mon[30713]: osdmap e5: 1 total, 0 up, 1 in 2024-02-07T13:32:51.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:51 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:32:52.165 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:51.853+0000 7fee07ebd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:32:52.165 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:51 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:51.950+0000 7fee07ebd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:32:52.166 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:32:52 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:32:52.165+0000 7fee07ebd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:32:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:52 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/4248979707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:32:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:52 smithi138 ceph-mon[30535]: Standby manager daemon x started 2024-02-07T13:32:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:52 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:32:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:52 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:32:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:52 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:32:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:52 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:32:52.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:52 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/4248979707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:32:52.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:52 smithi094 ceph-mon[25576]: Standby manager daemon x started 2024-02-07T13:32:52.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:52 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:32:52.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:52 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:32:52.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:52 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:32:52.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:52 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:32:52.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:52 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/4248979707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:32:52.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:52 smithi190 ceph-mon[30713]: Standby manager daemon x started 2024-02-07T13:32:52.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:52 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:32:52.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:52 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:32:52.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:52 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:32:52.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:52 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.138:0/3807858341' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:32:53.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:53 smithi094 ceph-mon[25576]: mgrmap e15: y(active, since 79s), standbys: z, x 2024-02-07T13:32:53.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:53 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:32:53.665 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:53 smithi094 ceph-mon[25576]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:53.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:53 smithi190 ceph-mon[30713]: mgrmap e15: y(active, since 79s), standbys: z, x 2024-02-07T13:32:53.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:53 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:32:53.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:53 smithi190 ceph-mon[30713]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:53.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:53 smithi138 ceph-mon[30535]: mgrmap e15: y(active, since 79s), standbys: z, x 2024-02-07T13:32:53.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:53 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:32:53.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:53 smithi138 ceph-mon[30535]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:55.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:55 smithi094 ceph-mon[25576]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:55.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:55 smithi190 ceph-mon[30713]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:56.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:55 smithi138 ceph-mon[30535]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:57.760 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:57 smithi094 ceph-mon[25576]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:58.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:57 smithi190 ceph-mon[30713]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:58.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:57 smithi138 ceph-mon[30535]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:32:59.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:58 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-07T13:32:59.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:58 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:59.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:58 smithi094 ceph-mon[25576]: Deploying daemon osd.0 on smithi094 2024-02-07T13:32:59.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:58 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-07T13:32:59.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:58 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:59.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:58 smithi190 ceph-mon[30713]: Deploying daemon osd.0 on smithi094 2024-02-07T13:32:59.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:58 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-07T13:32:59.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:58 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:32:59.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:58 smithi138 ceph-mon[30535]: Deploying daemon osd.0 on smithi094 2024-02-07T13:33:00.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:32:59 smithi094 ceph-mon[25576]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:00.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:32:59 smithi190 ceph-mon[30713]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:00.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:32:59 smithi138 ceph-mon[30535]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:02.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:01 smithi094 ceph-mon[25576]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:02.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:01 smithi190 ceph-mon[30713]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:02.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:01 smithi138 ceph-mon[30535]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:03.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:03 smithi094 ceph-mon[25576]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:03.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:03.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:03.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:04.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:03 smithi190 ceph-mon[30713]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:04.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:04.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:04.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:04.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:03 smithi138 ceph-mon[30535]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:04.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:04.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:04.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:06.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:05 smithi094 ceph-mon[25576]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:06.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:05 smithi190 ceph-mon[30713]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:06.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:05 smithi138 ceph-mon[30535]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:06.834 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 0 on host 'smithi094' 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:07.469 DEBUG:teuthology.orchestra.run.smithi094:osd.0> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.0.service 2024-02-07T13:33:07.471 INFO:tasks.cephadm:Deploying osd.1 on smithi138 with /dev/nvme4n1... 2024-02-07T13:33:07.472 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-07T13:33:07.501 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.501 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.502 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:07.502 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:07.502 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.502 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-07T13:33:07.502 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:07.502 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.502 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.502 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:07.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:07.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:07.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-07T13:33:07.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:07.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:07.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:07.714 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.b/config 2024-02-07T13:33:08.500 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:08 smithi094 ceph-mon[25576]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-07T13:33:08.500 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:08 smithi094 ceph-mon[25576]: osdmap e6: 1 total, 0 up, 1 in 2024-02-07T13:33:08.500 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:08 smithi094 ceph-mon[25576]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-07T13:33:08.500 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:08 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:08.528 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:08 smithi138 ceph-mon[30535]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-07T13:33:08.528 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:08 smithi138 ceph-mon[30535]: osdmap e6: 1 total, 0 up, 1 in 2024-02-07T13:33:08.528 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:08 smithi138 ceph-mon[30535]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-07T13:33:08.528 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:08 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:08.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:08 smithi190 ceph-mon[30713]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-07T13:33:08.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:08 smithi190 ceph-mon[30713]: osdmap e6: 1 total, 0 up, 1 in 2024-02-07T13:33:08.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:08 smithi190 ceph-mon[30713]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-07T13:33:08.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:08 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:08.763 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:33:08 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0[43272]: 2024-02-07T13:33:08.499+0000 7f1f35035700 -1 osd.0 0 waiting for initial osdmap 2024-02-07T13:33:08.763 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:33:08 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0[43272]: 2024-02-07T13:33:08.502+0000 7f1f2c626700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-07T13:33:09.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:09 smithi094 ceph-mon[25576]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-07T13:33:09.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:09 smithi094 ceph-mon[25576]: osdmap e7: 1 total, 0 up, 1 in 2024-02-07T13:33:09.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:09 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:09.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:09 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:09.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:09 smithi094 ceph-mon[25576]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' 2024-02-07T13:33:09.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:09 smithi094 ceph-mon[25576]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:09.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:09 smithi190 ceph-mon[30713]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-07T13:33:09.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:09 smithi190 ceph-mon[30713]: osdmap e7: 1 total, 0 up, 1 in 2024-02-07T13:33:09.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:09 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:09.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:09 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:09.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:09 smithi190 ceph-mon[30713]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' 2024-02-07T13:33:09.533 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:09 smithi190 ceph-mon[30713]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:09.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:09 smithi138 ceph-mon[30535]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-07T13:33:09.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:09 smithi138 ceph-mon[30535]: osdmap e7: 1 total, 0 up, 1 in 2024-02-07T13:33:09.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:09 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:09.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:09 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:09.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:09 smithi138 ceph-mon[30535]: from='osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933]' entity='osd.0' 2024-02-07T13:33:09.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:09 smithi138 ceph-mon[30535]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:33:09.647 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/nvme4n1 2024-02-07T13:33:09.647 INFO:teuthology.orchestra.run.smithi138.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-07T13:33:09.648 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-07T13:33:09.648 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-02-07T13:33:09.648 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-02-07T13:33:09.648 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127031 s, 825 MB/s 2024-02-07T13:33:09.648 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-02-07T13:33:10.282 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi138:/dev/nvme4n1 2024-02-07T13:33:10.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:10 smithi094 ceph-mon[25576]: purged_snaps scrub starts 2024-02-07T13:33:10.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:10 smithi094 ceph-mon[25576]: purged_snaps scrub ok 2024-02-07T13:33:10.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:10.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:10 smithi094 ceph-mon[25576]: osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933] boot 2024-02-07T13:33:10.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:10 smithi094 ceph-mon[25576]: osdmap e8: 1 total, 1 up, 1 in 2024-02-07T13:33:10.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:10.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:10 smithi190 ceph-mon[30713]: purged_snaps scrub starts 2024-02-07T13:33:10.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:10 smithi190 ceph-mon[30713]: purged_snaps scrub ok 2024-02-07T13:33:10.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:10 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:10.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:10 smithi190 ceph-mon[30713]: osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933] boot 2024-02-07T13:33:10.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:10 smithi190 ceph-mon[30713]: osdmap e8: 1 total, 1 up, 1 in 2024-02-07T13:33:10.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:10 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:10.557 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.b/config 2024-02-07T13:33:10.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:10 smithi138 ceph-mon[30535]: purged_snaps scrub starts 2024-02-07T13:33:10.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:10 smithi138 ceph-mon[30535]: purged_snaps scrub ok 2024-02-07T13:33:10.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:10 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:10.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:10 smithi138 ceph-mon[30535]: osd.0 [v2:172.21.15.94:6802/3542023933,v1:172.21.15.94:6803/3542023933] boot 2024-02-07T13:33:10.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:10 smithi138 ceph-mon[30535]: osdmap e8: 1 total, 1 up, 1 in 2024-02-07T13:33:10.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:10 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:33:11.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: Detected new or changed devices on smithi094 2024-02-07T13:33:11.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:11.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:11.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:33:11.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: Adjusting osd_memory_target on smithi094 to 17088M 2024-02-07T13:33:11.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:11.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:11.671 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:11.671 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:11.671 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:11.671 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:11 smithi138 ceph-mon[30535]: osdmap e9: 1 total, 1 up, 1 in 2024-02-07T13:33:12.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: Detected new or changed devices on smithi094 2024-02-07T13:33:12.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:12.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:12.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:33:12.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: Adjusting osd_memory_target on smithi094 to 17088M 2024-02-07T13:33:12.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:12.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:12.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:12.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:12.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:12.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:11 smithi094 ceph-mon[25576]: osdmap e9: 1 total, 1 up, 1 in 2024-02-07T13:33:12.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: Detected new or changed devices on smithi094 2024-02-07T13:33:12.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:12.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:12.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:33:12.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: Adjusting osd_memory_target on smithi094 to 17088M 2024-02-07T13:33:12.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:12.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:12.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:12.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:12.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:12.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:11 smithi190 ceph-mon[30713]: osdmap e9: 1 total, 1 up, 1 in 2024-02-07T13:33:12.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:12 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:33:12.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:12 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:33:12.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:12 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:12.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:12 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:33:12.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:12 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:33:12.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:12 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:13.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:12 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:33:13.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:12 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:33:13.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:12 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:13.922 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:13 smithi138 ceph-mon[30535]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:33:13.922 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:13 smithi138 ceph-mon[30535]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:14.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:13 smithi094 ceph-mon[25576]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:33:14.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:13 smithi094 ceph-mon[25576]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:14.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:13 smithi190 ceph-mon[30713]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:33:14.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:13 smithi190 ceph-mon[30713]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:16.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:15 smithi094 ceph-mon[25576]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:16.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:15 smithi094 ceph-mon[25576]: from='client.? 172.21.15.138:0/55192056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95884813-942c-4343-9aa4-ed974b154348"}]: dispatch 2024-02-07T13:33:16.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:15 smithi094 ceph-mon[25576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95884813-942c-4343-9aa4-ed974b154348"}]: dispatch 2024-02-07T13:33:16.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:15 smithi094 ceph-mon[25576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95884813-942c-4343-9aa4-ed974b154348"}]': finished 2024-02-07T13:33:16.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:15 smithi094 ceph-mon[25576]: osdmap e10: 2 total, 1 up, 2 in 2024-02-07T13:33:16.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:15 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:16.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:15 smithi190 ceph-mon[30713]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:16.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:15 smithi190 ceph-mon[30713]: from='client.? 172.21.15.138:0/55192056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95884813-942c-4343-9aa4-ed974b154348"}]: dispatch 2024-02-07T13:33:16.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:15 smithi190 ceph-mon[30713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95884813-942c-4343-9aa4-ed974b154348"}]: dispatch 2024-02-07T13:33:16.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:15 smithi190 ceph-mon[30713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95884813-942c-4343-9aa4-ed974b154348"}]': finished 2024-02-07T13:33:16.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:15 smithi190 ceph-mon[30713]: osdmap e10: 2 total, 1 up, 2 in 2024-02-07T13:33:16.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:15 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:16.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:15 smithi138 ceph-mon[30535]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:16.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:15 smithi138 ceph-mon[30535]: from='client.? 172.21.15.138:0/55192056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95884813-942c-4343-9aa4-ed974b154348"}]: dispatch 2024-02-07T13:33:16.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:15 smithi138 ceph-mon[30535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95884813-942c-4343-9aa4-ed974b154348"}]: dispatch 2024-02-07T13:33:16.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:15 smithi138 ceph-mon[30535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95884813-942c-4343-9aa4-ed974b154348"}]': finished 2024-02-07T13:33:16.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:15 smithi138 ceph-mon[30535]: osdmap e10: 2 total, 1 up, 2 in 2024-02-07T13:33:16.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:15 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:17.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:16 smithi094 ceph-mon[25576]: from='client.? 172.21.15.138:0/474371190' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:33:17.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:16 smithi190 ceph-mon[30713]: from='client.? 172.21.15.138:0/474371190' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:33:17.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:16 smithi138 ceph-mon[30535]: from='client.? 172.21.15.138:0/474371190' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:33:18.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:17 smithi094 ceph-mon[25576]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:18.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:17 smithi190 ceph-mon[30713]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:18.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:17 smithi138 ceph-mon[30535]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:19.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:19 smithi138 ceph-mon[30535]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:19.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:19 smithi094 ceph-mon[25576]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:19.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:19 smithi190 ceph-mon[30713]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:22.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:21 smithi094 ceph-mon[25576]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:22.025 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:21 smithi138 ceph-mon[30535]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:22.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:21 smithi190 ceph-mon[30713]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:23.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T13:33:23.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:22 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:23.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T13:33:23.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:22 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:23.060 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:22 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T13:33:23.060 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:22 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:23.899 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:23 smithi138 ceph-mon[30535]: Deploying daemon osd.1 on smithi138 2024-02-07T13:33:23.900 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:23 smithi138 ceph-mon[30535]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:23 smithi094 ceph-mon[25576]: Deploying daemon osd.1 on smithi138 2024-02-07T13:33:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:23 smithi094 ceph-mon[25576]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:23 smithi190 ceph-mon[30713]: Deploying daemon osd.1 on smithi138 2024-02-07T13:33:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:23 smithi190 ceph-mon[30713]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:25.891 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:25 smithi138 ceph-mon[30535]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:26.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:25 smithi094 ceph-mon[25576]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:26.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:25 smithi190 ceph-mon[30713]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:27.837 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:27 smithi138 ceph-mon[30535]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:28.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:27 smithi094 ceph-mon[25576]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:28.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:27 smithi190 ceph-mon[30713]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:28.908 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:28 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:28.908 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:28 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:28.908 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:28 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:28.908 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:28 smithi138 ceph-mon[30535]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:29.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:28 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:29.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:28 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:29.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:28 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:29.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:28 smithi094 ceph-mon[25576]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:29.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:28 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:29.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:28 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:29.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:28 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:29.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:28 smithi190 ceph-mon[30713]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:31.765 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:31 smithi138 ceph-mon[30535]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:31.765 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:31 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:31.765 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:31 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:31.765 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:31 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:31.765 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:31 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:31.765 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:31 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:31.766 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:31 smithi138 ceph-mon[30535]: from='osd.1 [v2:172.21.15.138:6800/372706022,v1:172.21.15.138:6801/372706022]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:33:31.766 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:31 smithi138 ceph-mon[30535]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:33:31.812 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 1 on host 'smithi138' 2024-02-07T13:33:32.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:31 smithi094 ceph-mon[25576]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:32.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:32.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:32.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:32.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:32.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:32.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:31 smithi094 ceph-mon[25576]: from='osd.1 [v2:172.21.15.138:6800/372706022,v1:172.21.15.138:6801/372706022]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:33:32.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:31 smithi094 ceph-mon[25576]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:33:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:31 smithi190 ceph-mon[30713]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:31 smithi190 ceph-mon[30713]: from='osd.1 [v2:172.21.15.138:6800/372706022,v1:172.21.15.138:6801/372706022]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:33:32.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:31 smithi190 ceph-mon[30713]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:33:32.404 DEBUG:teuthology.orchestra.run.smithi138:osd.1> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.1.service 2024-02-07T13:33:32.407 INFO:tasks.cephadm:Deploying osd.2 on smithi190 with /dev/nvme4n1... 2024-02-07T13:33:32.407 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-07T13:33:32.649 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:33:33.017 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:32 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:33.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:32 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:33.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:32 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:33.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:32 smithi190 ceph-mon[30713]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-07T13:33:33.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:32 smithi190 ceph-mon[30713]: osdmap e11: 2 total, 1 up, 2 in 2024-02-07T13:33:33.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:32 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:33.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:32 smithi190 ceph-mon[30713]: from='osd.1 [v2:172.21.15.138:6800/372706022,v1:172.21.15.138:6801/372706022]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-02-07T13:33:33.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:32 smithi190 ceph-mon[30713]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-02-07T13:33:33.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:32 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:33.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:32 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:33.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:32 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:33.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:32 smithi138 ceph-mon[30535]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-07T13:33:33.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:32 smithi138 ceph-mon[30535]: osdmap e11: 2 total, 1 up, 2 in 2024-02-07T13:33:33.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:32 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:33.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:32 smithi138 ceph-mon[30535]: from='osd.1 [v2:172.21.15.138:6800/372706022,v1:172.21.15.138:6801/372706022]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-02-07T13:33:33.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:32 smithi138 ceph-mon[30535]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-02-07T13:33:33.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:33.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:33.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:33.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:32 smithi094 ceph-mon[25576]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-07T13:33:33.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:32 smithi094 ceph-mon[25576]: osdmap e11: 2 total, 1 up, 2 in 2024-02-07T13:33:33.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:33.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:32 smithi094 ceph-mon[25576]: from='osd.1 [v2:172.21.15.138:6800/372706022,v1:172.21.15.138:6801/372706022]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-02-07T13:33:33.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:32 smithi094 ceph-mon[25576]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-02-07T13:33:33.568 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:33:33 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1[35112]: 2024-02-07T13:33:33.366+0000 7fa28c4b2700 -1 osd.1 0 waiting for initial osdmap 2024-02-07T13:33:33.568 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:33:33 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1[35112]: 2024-02-07T13:33:33.368+0000 7fa2842a4700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-07T13:33:33.819 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:33 smithi138 ceph-mon[30535]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:33 smithi138 ceph-mon[30535]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-02-07T13:33:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:33 smithi138 ceph-mon[30535]: osdmap e12: 2 total, 1 up, 2 in 2024-02-07T13:33:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:33 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:33 smithi138 ceph-mon[30535]: from='osd.1 ' entity='osd.1' 2024-02-07T13:33:33.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:33 smithi190 ceph-mon[30713]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:33.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:33 smithi190 ceph-mon[30713]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-02-07T13:33:33.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:33 smithi190 ceph-mon[30713]: osdmap e12: 2 total, 1 up, 2 in 2024-02-07T13:33:33.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:33 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:33.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:33 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:33.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:33 smithi190 ceph-mon[30713]: from='osd.1 ' entity='osd.1' 2024-02-07T13:33:34.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:33 smithi094 ceph-mon[25576]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:34.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:33 smithi094 ceph-mon[25576]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-02-07T13:33:34.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:33 smithi094 ceph-mon[25576]: osdmap e12: 2 total, 1 up, 2 in 2024-02-07T13:33:34.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:33 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:34.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:33 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:34.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:33 smithi094 ceph-mon[25576]: from='osd.1 ' entity='osd.1' 2024-02-07T13:33:34.545 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/nvme4n1 2024-02-07T13:33:34.545 INFO:teuthology.orchestra.run.smithi190.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-07T13:33:34.545 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-07T13:33:34.545 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-02-07T13:33:34.546 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-02-07T13:33:34.546 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0141164 s, 743 MB/s 2024-02-07T13:33:34.546 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-02-07T13:33:35.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:34 smithi094 ceph-mon[25576]: purged_snaps scrub starts 2024-02-07T13:33:35.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:34 smithi094 ceph-mon[25576]: purged_snaps scrub ok 2024-02-07T13:33:35.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:35.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:34 smithi094 ceph-mon[25576]: osd.1 [v2:172.21.15.138:6800/372706022,v1:172.21.15.138:6801/372706022] boot 2024-02-07T13:33:35.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:34 smithi094 ceph-mon[25576]: osdmap e13: 2 total, 2 up, 2 in 2024-02-07T13:33:35.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:34 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:35.068 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:34 smithi190 ceph-mon[30713]: purged_snaps scrub starts 2024-02-07T13:33:35.068 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:34 smithi190 ceph-mon[30713]: purged_snaps scrub ok 2024-02-07T13:33:35.068 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:35.068 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:34 smithi190 ceph-mon[30713]: osd.1 [v2:172.21.15.138:6800/372706022,v1:172.21.15.138:6801/372706022] boot 2024-02-07T13:33:35.068 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:34 smithi190 ceph-mon[30713]: osdmap e13: 2 total, 2 up, 2 in 2024-02-07T13:33:35.068 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:34 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:35.069 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi190:/dev/nvme4n1 2024-02-07T13:33:35.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:34 smithi138 ceph-mon[30535]: purged_snaps scrub starts 2024-02-07T13:33:35.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:34 smithi138 ceph-mon[30535]: purged_snaps scrub ok 2024-02-07T13:33:35.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:35.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:34 smithi138 ceph-mon[30535]: osd.1 [v2:172.21.15.138:6800/372706022,v1:172.21.15.138:6801/372706022] boot 2024-02-07T13:33:35.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:34 smithi138 ceph-mon[30535]: osdmap e13: 2 total, 2 up, 2 in 2024-02-07T13:33:35.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:34 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:33:35.301 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:33:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:35 smithi190 ceph-mon[30713]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:35 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:35 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:35 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:33:36.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:35 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:35 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:36.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:35 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:36.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:35 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:35 smithi138 ceph-mon[30535]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:35 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:35 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:35 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:33:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:35 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:35 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:35 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:35 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:35 smithi094 ceph-mon[25576]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:33:36.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:33:36.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:36.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:36.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:36.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:35 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:37.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:36 smithi190 ceph-mon[30713]: Detected new or changed devices on smithi138 2024-02-07T13:33:37.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:36 smithi190 ceph-mon[30713]: Adjusting osd_memory_target on smithi138 to 17088M 2024-02-07T13:33:37.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:36 smithi190 ceph-mon[30713]: osdmap e14: 2 total, 2 up, 2 in 2024-02-07T13:33:37.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:36 smithi138 ceph-mon[30535]: Detected new or changed devices on smithi138 2024-02-07T13:33:37.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:36 smithi138 ceph-mon[30535]: Adjusting osd_memory_target on smithi138 to 17088M 2024-02-07T13:33:37.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:36 smithi138 ceph-mon[30535]: osdmap e14: 2 total, 2 up, 2 in 2024-02-07T13:33:37.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:36 smithi094 ceph-mon[25576]: Detected new or changed devices on smithi138 2024-02-07T13:33:37.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:36 smithi094 ceph-mon[25576]: Adjusting osd_memory_target on smithi138 to 17088M 2024-02-07T13:33:37.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:36 smithi094 ceph-mon[25576]: osdmap e14: 2 total, 2 up, 2 in 2024-02-07T13:33:37.870 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:37 smithi190 ceph-mon[30713]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:37.871 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:37 smithi190 ceph-mon[30713]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:33:37.871 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:37 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:33:37.871 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:37 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:33:37.871 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:37 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:38.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:37 smithi138 ceph-mon[30535]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:38.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:37 smithi138 ceph-mon[30535]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:33:38.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:37 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:33:38.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:37 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:33:38.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:37 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:38.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:37 smithi094 ceph-mon[25576]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:38.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:37 smithi094 ceph-mon[25576]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:33:38.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:37 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:33:38.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:37 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:33:38.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:37 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:39.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:39 smithi094 ceph-mon[25576]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:39.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:39 smithi190 ceph-mon[30713]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:39.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:39 smithi138 ceph-mon[30535]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:40 smithi094 ceph-mon[25576]: from='client.? 172.21.15.190:0/3116642483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "770fe04a-021a-49a9-b88b-5f4ab50822b3"}]: dispatch 2024-02-07T13:33:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:40 smithi094 ceph-mon[25576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "770fe04a-021a-49a9-b88b-5f4ab50822b3"}]: dispatch 2024-02-07T13:33:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:40 smithi094 ceph-mon[25576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "770fe04a-021a-49a9-b88b-5f4ab50822b3"}]': finished 2024-02-07T13:33:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:40 smithi094 ceph-mon[25576]: osdmap e15: 3 total, 2 up, 3 in 2024-02-07T13:33:40.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:40 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:40.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:40 smithi190 ceph-mon[30713]: from='client.? 172.21.15.190:0/3116642483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "770fe04a-021a-49a9-b88b-5f4ab50822b3"}]: dispatch 2024-02-07T13:33:40.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:40 smithi190 ceph-mon[30713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "770fe04a-021a-49a9-b88b-5f4ab50822b3"}]: dispatch 2024-02-07T13:33:40.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:40 smithi190 ceph-mon[30713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "770fe04a-021a-49a9-b88b-5f4ab50822b3"}]': finished 2024-02-07T13:33:40.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:40 smithi190 ceph-mon[30713]: osdmap e15: 3 total, 2 up, 3 in 2024-02-07T13:33:40.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:40 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:40.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:40 smithi138 ceph-mon[30535]: from='client.? 172.21.15.190:0/3116642483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "770fe04a-021a-49a9-b88b-5f4ab50822b3"}]: dispatch 2024-02-07T13:33:40.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:40 smithi138 ceph-mon[30535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "770fe04a-021a-49a9-b88b-5f4ab50822b3"}]: dispatch 2024-02-07T13:33:40.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:40 smithi138 ceph-mon[30535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "770fe04a-021a-49a9-b88b-5f4ab50822b3"}]': finished 2024-02-07T13:33:40.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:40 smithi138 ceph-mon[30535]: osdmap e15: 3 total, 2 up, 3 in 2024-02-07T13:33:40.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:40 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:41.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:41 smithi094 ceph-mon[25576]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:41.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:41 smithi094 ceph-mon[25576]: from='client.? 172.21.15.190:0/962132778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:33:41.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:41 smithi190 ceph-mon[30713]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:41.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:41 smithi190 ceph-mon[30713]: from='client.? 172.21.15.190:0/962132778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:33:41.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:41 smithi138 ceph-mon[30535]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:41.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:41 smithi138 ceph-mon[30535]: from='client.? 172.21.15.190:0/962132778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:33:43.836 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:43 smithi190 ceph-mon[30713]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:44.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:43 smithi094 ceph-mon[25576]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:44.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:43 smithi138 ceph-mon[30535]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:45.894 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:45 smithi190 ceph-mon[30713]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:46.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:45 smithi094 ceph-mon[25576]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:46.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:45 smithi138 ceph-mon[30535]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:47.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:46 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-07T13:33:47.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:46 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:47.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:46 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-07T13:33:47.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:46 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:47.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:46 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-07T13:33:47.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:46 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:47.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:47 smithi190 ceph-mon[30713]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:47.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:47 smithi190 ceph-mon[30713]: Deploying daemon osd.2 on smithi190 2024-02-07T13:33:48.015 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:47 smithi094 ceph-mon[25576]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:48.015 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:47 smithi094 ceph-mon[25576]: Deploying daemon osd.2 on smithi190 2024-02-07T13:33:48.092 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:47 smithi138 ceph-mon[30535]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:48.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:47 smithi138 ceph-mon[30535]: Deploying daemon osd.2 on smithi190 2024-02-07T13:33:49.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:49 smithi190 ceph-mon[30713]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:49.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:49 smithi094 ceph-mon[25576]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:49.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:49 smithi138 ceph-mon[30535]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:51.962 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:51 smithi190 ceph-mon[30713]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:52.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:51 smithi094 ceph-mon[25576]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:52.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:51 smithi138 ceph-mon[30535]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:53.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:53 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:53.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:53 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:53.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:53 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:53.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:53 smithi190 ceph-mon[30713]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:53.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:53 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:53.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:53 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:53.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:53 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:53.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:53 smithi094 ceph-mon[25576]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:53.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:53 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:53.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:53 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:53.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:53 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:53.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:53 smithi138 ceph-mon[30535]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:55.589 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 2 on host 'smithi190' 2024-02-07T13:33:55.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:55.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:55.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:55.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:55.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:55.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:55.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='osd.2 [v2:172.21.15.190:6800/2499326806,v1:172.21.15.190:6801/2499326806]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:33:55.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:33:55.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:55.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:55.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:55 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:56.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:56.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:56.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:56.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='osd.2 [v2:172.21.15.190:6800/2499326806,v1:172.21.15.190:6801/2499326806]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:33:56.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:33:56.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:55 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:56.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:56.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:33:56.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:33:56.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='osd.2 [v2:172.21.15.190:6800/2499326806,v1:172.21.15.190:6801/2499326806]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:33:56.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:33:56.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:33:56.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:55 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:33:56.319 DEBUG:teuthology.orchestra.run.smithi190:osd.2> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.2.service 2024-02-07T13:33:56.321 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-02-07T13:33:56.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph osd stat -f json 2024-02-07T13:33:56.562 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:33:57.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:57 smithi094 ceph-mon[25576]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-07T13:33:57.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:57 smithi094 ceph-mon[25576]: osdmap e16: 3 total, 2 up, 3 in 2024-02-07T13:33:57.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:57 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:57.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:57 smithi094 ceph-mon[25576]: from='osd.2 [v2:172.21.15.190:6800/2499326806,v1:172.21.15.190:6801/2499326806]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-02-07T13:33:57.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:57 smithi094 ceph-mon[25576]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-02-07T13:33:57.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:57 smithi094 ceph-mon[25576]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:57.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:57 smithi190 ceph-mon[30713]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-07T13:33:57.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:57 smithi190 ceph-mon[30713]: osdmap e16: 3 total, 2 up, 3 in 2024-02-07T13:33:57.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:57 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:57.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:57 smithi190 ceph-mon[30713]: from='osd.2 [v2:172.21.15.190:6800/2499326806,v1:172.21.15.190:6801/2499326806]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-02-07T13:33:57.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:57 smithi190 ceph-mon[30713]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-02-07T13:33:57.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:57 smithi190 ceph-mon[30713]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:57.281 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:33:57 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2[35992]: 2024-02-07T13:33:57.015+0000 7fdbbe370700 -1 osd.2 0 waiting for initial osdmap 2024-02-07T13:33:57.281 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:33:57 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2[35992]: 2024-02-07T13:33:57.018+0000 7fdbba16a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-07T13:33:57.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:57 smithi138 ceph-mon[30535]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-07T13:33:57.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:57 smithi138 ceph-mon[30535]: osdmap e16: 3 total, 2 up, 3 in 2024-02-07T13:33:57.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:57 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:57.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:57 smithi138 ceph-mon[30535]: from='osd.2 [v2:172.21.15.190:6800/2499326806,v1:172.21.15.190:6801/2499326806]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-02-07T13:33:57.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:57 smithi138 ceph-mon[30535]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-02-07T13:33:57.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:57 smithi138 ceph-mon[30535]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:58.121 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:58 smithi094 ceph-mon[25576]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-02-07T13:33:58.122 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:58 smithi094 ceph-mon[25576]: osdmap e17: 3 total, 2 up, 3 in 2024-02-07T13:33:58.122 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:58 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:58.122 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:58 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:58.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:58 smithi190 ceph-mon[30713]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-02-07T13:33:58.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:58 smithi190 ceph-mon[30713]: osdmap e17: 3 total, 2 up, 3 in 2024-02-07T13:33:58.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:58 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:58.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:58 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:58.320 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:33:58.342 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:58 smithi138 ceph-mon[30535]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-02-07T13:33:58.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:58 smithi138 ceph-mon[30535]: osdmap e17: 3 total, 2 up, 3 in 2024-02-07T13:33:58.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:58 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:58.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:58 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:58.917 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":18,"num_osds":3,"num_up_osds":3,"osd_up_since":1707312838,"num_in_osds":3,"osd_in_since":1707312819,"num_remapped_pgs":0} 2024-02-07T13:33:58.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-07T13:33:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:33:59.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:59 smithi190 ceph-mon[30713]: purged_snaps scrub starts 2024-02-07T13:33:59.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:59 smithi190 ceph-mon[30713]: purged_snaps scrub ok 2024-02-07T13:33:59.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:59 smithi190 ceph-mon[30713]: osd.2 [v2:172.21.15.190:6800/2499326806,v1:172.21.15.190:6801/2499326806] boot 2024-02-07T13:33:59.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:59 smithi190 ceph-mon[30713]: osdmap e18: 3 total, 3 up, 3 in 2024-02-07T13:33:59.203 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:59 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:59.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:59 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/768652442' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T13:33:59.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:59 smithi190 ceph-mon[30713]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:59.204 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:33:59 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:33:59.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:59 smithi094 ceph-mon[25576]: purged_snaps scrub starts 2024-02-07T13:33:59.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:59 smithi094 ceph-mon[25576]: purged_snaps scrub ok 2024-02-07T13:33:59.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:59 smithi094 ceph-mon[25576]: osd.2 [v2:172.21.15.190:6800/2499326806,v1:172.21.15.190:6801/2499326806] boot 2024-02-07T13:33:59.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:59 smithi094 ceph-mon[25576]: osdmap e18: 3 total, 3 up, 3 in 2024-02-07T13:33:59.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:59 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:59.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:59 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/768652442' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T13:33:59.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:59 smithi094 ceph-mon[25576]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:59.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:33:59 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:33:59.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:59 smithi138 ceph-mon[30535]: purged_snaps scrub starts 2024-02-07T13:33:59.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:59 smithi138 ceph-mon[30535]: purged_snaps scrub ok 2024-02-07T13:33:59.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:59 smithi138 ceph-mon[30535]: osd.2 [v2:172.21.15.190:6800/2499326806,v1:172.21.15.190:6801/2499326806] boot 2024-02-07T13:33:59.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:59 smithi138 ceph-mon[30535]: osdmap e18: 3 total, 3 up, 3 in 2024-02-07T13:33:59.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:59 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:33:59.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:59 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/768652442' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T13:33:59.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:59 smithi138 ceph-mon[30535]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:33:59.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:33:59 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:34:00.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: osdmap e19: 3 total, 3 up, 3 in 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: Detected new or changed devices on smithi190 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: Adjusting osd_memory_target on smithi190 to 17088M 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:00.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:00 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: osdmap e19: 3 total, 3 up, 3 in 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: Detected new or changed devices on smithi190 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: Adjusting osd_memory_target on smithi190 to 17088M 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:00.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:00 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-07T13:34:00.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: osdmap e19: 3 total, 3 up, 3 in 2024-02-07T13:34:00.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:34:00.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: Detected new or changed devices on smithi190 2024-02-07T13:34:00.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:34:00.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: Adjusting osd_memory_target on smithi190 to 17088M 2024-02-07T13:34:00.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:00.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:00.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:00 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:00.603 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:34:00 smithi094 sudo[48679]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-07T13:34:00.603 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:34:00 smithi094 sudo[48679]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:00.603 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:34:00 smithi094 sudo[48679]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:00.603 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:34:00 smithi094 sudo[48679]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:01.013 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:34:00 smithi094 sudo[48713]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-07T13:34:01.013 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:34:00 smithi094 sudo[48713]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:01.013 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:34:00 smithi094 sudo[48713]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:01.013 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:34:00 smithi094 sudo[48713]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:01.033 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:34:01.047 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"fsid":"edf33ada-c5bc-11ee-95b6-87774f69a715","created":"2024-02-07T13:30:33.379810+0000","modified":"2024-02-07T13:34:01.029372+0000","last_up_change":"2024-02-07T13:33:58.013510+0000","last_in_change":"2024-02-07T13:33:39.700912+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-07T13:33:58.966796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a54e30c-a30a-4824-be90-1326fb7d73fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6803","nonce":3542023933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6805","nonce":3542023933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6809","nonce":3542023933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6807","nonce":3542023933}]},"public_addr":"172.21.15.94:6803/3542023933","cluster_addr":"172.21.15.94:6805/3542023933","heartbeat_back_addr":"172.21.15.94:6809/3542023933","heartbeat_front_addr":"172.21.15.94:6807/3542023933","state":["exists","up"]},{"osd":1,"uuid":"95884813-942c-4343-9aa4-ed974b154348","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6801","nonce":372706022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6803","nonce":372706022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6807","nonce":372706022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6805","nonce":372706022}]},"public_addr":"172.21.15.138:6801/372706022","cluster_addr":"172.21.15.138:6803/372706022","heartbeat_back_addr":"172.21.15.138:6807/372706022","heartbeat_front_addr":"172.21.15.138:6805/372706022","state":["exists","up"]},{"osd":2,"uuid":"770fe04a-021a-49a9-b88b-5f4ab50822b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6801","nonce":2499326806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6803","nonce":2499326806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6807","nonce":2499326806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6805","nonce":2499326806}]},"public_addr":"172.21.15.190:6801/2499326806","cluster_addr":"172.21.15.190:6803/2499326806","heartbeat_back_addr":"172.21.15.190:6807/2499326806","heartbeat_front_addr":"172.21.15.190:6805/2499326806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:33:07.338337+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:33:32.050787+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:33:56.163828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/3836439501":"2024-02-08T13:31:32.635798+0000","172.21.15.94:6800/3836439501":"2024-02-08T13:31:32.635798+0000","172.21.15.94:0/2055038774":"2024-02-08T13:31:32.635798+0000","172.21.15.94:6801/2086851078":"2024-02-08T13:31:06.303667+0000","172.21.15.94:6800/2086851078":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/3678689078":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/1966097319":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/1166789205":"2024-02-08T13:31:32.635798+0000","172.21.15.94:0/43997712":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/2217401373":"2024-02-08T13:31:32.635798+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-07T13:34:01.153 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:01 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-07T13:34:01.153 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:01 smithi138 ceph-mon[30535]: osdmap e20: 3 total, 3 up, 3 in 2024-02-07T13:34:01.153 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:01 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:01.153 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:01 smithi138 ceph-mon[30535]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:01.154 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:34:00 smithi138 sudo[40008]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-07T13:34:01.154 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:34:00 smithi138 sudo[40008]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:01.154 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:34:00 smithi138 sudo[40008]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:01.154 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:34:00 smithi138 sudo[40008]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:01.339 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:01 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-07T13:34:01.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:01 smithi094 ceph-mon[25576]: osdmap e20: 3 total, 3 up, 3 in 2024-02-07T13:34:01.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:01 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:01.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:01 smithi094 ceph-mon[25576]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:01.419 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:01 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-07T13:34:01.419 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:01 smithi190 ceph-mon[30713]: osdmap e20: 3 total, 3 up, 3 in 2024-02-07T13:34:01.419 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:01 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:01.420 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:01 smithi190 ceph-mon[30713]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:01.593 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:34:01 smithi138 sudo[40011]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-07T13:34:01.593 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:34:01 smithi138 sudo[40011]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:01.593 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:34:01 smithi138 sudo[40011]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:01.593 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:34:01 smithi138 sudo[40011]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:01.656 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-07T13:33:58.966796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-07T13:34:01.656 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-07T13:34:01.669 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:34:01 smithi190 sudo[40794]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-07T13:34:01.670 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:34:01 smithi190 sudo[40794]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:01.670 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:34:01 smithi190 sudo[40794]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:01.670 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:34:01 smithi190 sudo[40794]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:01.670 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:34:01 smithi190 sudo[40797]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-07T13:34:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:02.030 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:34:01 smithi190 sudo[40797]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:02.030 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:34:01 smithi190 sudo[40797]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:02.030 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:34:01 smithi190 sudo[40797]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:02.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:01 smithi094 sudo[48845]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-07T13:34:02.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:01 smithi094 sudo[48845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:02.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:01 smithi094 sudo[48845]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:02.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:01 smithi094 sudo[48845]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:02.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 ceph-mon[30713]: osdmap e21: 3 total, 3 up, 3 in 2024-02-07T13:34:02.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/284381854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:34:02.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:02.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:02.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 ceph-mon[30713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:34:02.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:02 smithi094 sudo[48867]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-07T13:34:02.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:02 smithi094 sudo[48867]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:02.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:02 smithi094 sudo[48867]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:02.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:02 smithi094 sudo[48867]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:02.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:02 smithi094 ceph-mon[25576]: osdmap e21: 3 total, 3 up, 3 in 2024-02-07T13:34:02.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:02 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/284381854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:34:02.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:02 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:02.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:02 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:02.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:02 smithi094 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:34:02.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:02 smithi138 ceph-mon[30535]: osdmap e21: 3 total, 3 up, 3 in 2024-02-07T13:34:02.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:02 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/284381854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:34:02.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:02 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:02.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:02 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:02.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:02 smithi138 ceph-mon[30535]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:34:02.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 sudo[40800]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-07T13:34:02.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 sudo[40800]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:02.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 sudo[40800]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:02.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 sudo[40800]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:03.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 sudo[40803]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-07T13:34:03.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 sudo[40803]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:03.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 sudo[40803]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:03.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:02 smithi190 sudo[40803]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:03.247 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:02 smithi138 sudo[40014]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-07T13:34:03.248 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:02 smithi138 sudo[40014]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:03.248 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:02 smithi138 sudo[40014]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:03.248 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 sudo[40014]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:03.490 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:03.491 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:34:03.491 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:03.491 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:03.491 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:03.491 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:03 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:03.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:34:03.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:03.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:03.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:03.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:03 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:03.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 sudo[40017]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-07T13:34:03.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 sudo[40017]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:34:03.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 sudo[40017]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 sudo[40017]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:03.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:03.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:03.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:03.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:34:03.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:03.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:03.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:03.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:03 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:03.727 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-07T13:34:04.317 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:04 smithi094 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:34:04.317 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:04 smithi094 ceph-mon[25576]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:34:04.317 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:04 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:04.317 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:04 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3676749049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-07T13:34:04.342 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi094 2024-02-07T13:34:04.343 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi094=foo.a' 2024-02-07T13:34:04.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:04 smithi190 ceph-mon[30713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:34:04.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:04 smithi190 ceph-mon[30713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:34:04.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:04 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:04.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:04 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3676749049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-07T13:34:04.590 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:34:04.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:04 smithi138 ceph-mon[30535]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:34:04.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:04 smithi138 ceph-mon[30535]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:34:04.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:04 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:04.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:04 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3676749049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-07T13:34:05.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:05 smithi094 ceph-mon[25576]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:05.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:05 smithi190 ceph-mon[30713]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:05.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:05 smithi138 ceph-mon[30535]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:06.392 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled rgw.foo.a update... 2024-02-07T13:34:07.003 DEBUG:teuthology.orchestra.run.smithi094:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@rgw.foo.a.service 2024-02-07T13:34:07.006 INFO:tasks.cephadm:Adding prometheus.a on smithi138 2024-02-07T13:34:07.006 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch apply prometheus '1;smithi138=a' 2024-02-07T13:34:07.240 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi094=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: Saving service rgw.foo.a spec with placement smithi094=foo.a;count:1 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-07T13:34:07.618 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-07T13:34:07.619 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:07.619 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:07 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi094=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: Saving service rgw.foo.a spec with placement smithi094=foo.a;count:1 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-07T13:34:07.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-07T13:34:07.723 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:07.723 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:07 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:07.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi094=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:07.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: Saving service rgw.foo.a spec with placement smithi094=foo.a;count:1 2024-02-07T13:34:07.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:07.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:07.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:07.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:07.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:07.844 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:07.844 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-07T13:34:07.844 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-07T13:34:07.844 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:07.844 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:07 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:08.598 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:08 smithi094 ceph-mon[25576]: Deploying daemon rgw.foo.a on smithi094 2024-02-07T13:34:08.623 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:08 smithi190 ceph-mon[30713]: Deploying daemon rgw.foo.a on smithi094 2024-02-07T13:34:08.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:08 smithi138 ceph-mon[30535]: Deploying daemon rgw.foo.a on smithi094 2024-02-07T13:34:09.054 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled prometheus update... 2024-02-07T13:34:09.258 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:34:09 smithi094 systemd[1]: Starting Ceph rgw.foo.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:34:09.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:09 smithi094 ceph-mon[25576]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:09.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:09 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:09.513 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:34:09 smithi094 podman[49298]: 2024-02-07 13:34:09.161441345 +0000 UTC m=+0.017547241 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:34:09.513 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:34:09 smithi094 podman[49298]: 2024-02-07 13:34:09.267598868 +0000 UTC m=+0.123704797 container create b518cef04ad6a897b8ec1859337cf27a64be62a02f327e4b7ff04fe9059badab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-rgw-foo-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T13:34:09.513 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:34:09 smithi094 podman[49298]: 2024-02-07 13:34:09.494995555 +0000 UTC m=+0.351101455 container init b518cef04ad6a897b8ec1859337cf27a64be62a02f327e4b7ff04fe9059badab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-rgw-foo-a, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, ceph=True) 2024-02-07T13:34:09.514 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:34:09 smithi094 podman[49298]: 2024-02-07 13:34:09.500240357 +0000 UTC m=+0.356346249 container start b518cef04ad6a897b8ec1859337cf27a64be62a02f327e4b7ff04fe9059badab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:34:09.610 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:09 smithi190 ceph-mon[30713]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:09.610 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:09 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:09.639 DEBUG:teuthology.orchestra.run.smithi138:prometheus.a> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@prometheus.a.service 2024-02-07T13:34:09.642 INFO:tasks.cephadm:Adding node-exporter.a on smithi094 2024-02-07T13:34:09.642 INFO:tasks.cephadm:Adding node-exporter.b on smithi138 2024-02-07T13:34:09.642 INFO:tasks.cephadm:Adding node-exporter.c on smithi190 2024-02-07T13:34:09.642 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch apply node-exporter '3;smithi094=a;smithi138=b;smithi190=c' 2024-02-07T13:34:09.669 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:09 smithi138 ceph-mon[30535]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:09.669 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:09 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:09.875 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:34:10.013 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:34:09 smithi094 bash[49298]: b518cef04ad6a897b8ec1859337cf27a64be62a02f327e4b7ff04fe9059badab 2024-02-07T13:34:10.013 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:34:09 smithi094 systemd[1]: Started Ceph rgw.foo.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:34:11.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi138=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:11.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: Saving service prometheus spec with placement smithi138=a;count:1 2024-02-07T13:34:11.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: Saving service rgw.foo.a spec with placement smithi094=foo.a;count:1 2024-02-07T13:34:11.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:11.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: osdmap e22: 3 total, 3 up, 3 in 2024-02-07T13:34:11.014 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:10 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2599732716' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-07T13:34:11.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi138=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:11.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: Saving service prometheus spec with placement smithi138=a;count:1 2024-02-07T13:34:11.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: Saving service rgw.foo.a spec with placement smithi094=foo.a;count:1 2024-02-07T13:34:11.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:11.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: osdmap e22: 3 total, 3 up, 3 in 2024-02-07T13:34:11.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:10 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/2599732716' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-07T13:34:11.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi138=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:11.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: Saving service prometheus spec with placement smithi138=a;count:1 2024-02-07T13:34:11.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: Saving service rgw.foo.a spec with placement smithi094=foo.a;count:1 2024-02-07T13:34:11.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:11.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:11.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: osdmap e22: 3 total, 3 up, 3 in 2024-02-07T13:34:11.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:10 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/2599732716' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-07T13:34:11.720 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled node-exporter update... 2024-02-07T13:34:12.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:11 smithi094 ceph-mon[25576]: pgmap v91: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:12.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:12.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:12.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:12.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:12.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:11 smithi094 ceph-mon[25576]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T13:34:12.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:11 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2599732716' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-07T13:34:12.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:11 smithi094 ceph-mon[25576]: osdmap e23: 3 total, 3 up, 3 in 2024-02-07T13:34:12.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:11 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:12.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:11 smithi190 ceph-mon[30713]: pgmap v91: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:12.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:12.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:12.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:12.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:12.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:11 smithi190 ceph-mon[30713]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T13:34:12.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:11 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/2599732716' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-07T13:34:12.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:11 smithi190 ceph-mon[30713]: osdmap e23: 3 total, 3 up, 3 in 2024-02-07T13:34:12.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:11 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:12.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:12 smithi138 ceph-mon[30535]: pgmap v91: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:12.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:12 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:12.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:12 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:12.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:12 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:12.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:12 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:12.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:12 smithi138 ceph-mon[30535]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T13:34:12.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:12 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/2599732716' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-07T13:34:12.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:12 smithi138 ceph-mon[30535]: osdmap e23: 3 total, 3 up, 3 in 2024-02-07T13:34:12.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:12 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:12.352 DEBUG:teuthology.orchestra.run.smithi094:node-exporter.a> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@node-exporter.a.service 2024-02-07T13:34:12.354 DEBUG:teuthology.orchestra.run.smithi138:node-exporter.b> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@node-exporter.b.service 2024-02-07T13:34:12.357 DEBUG:teuthology.orchestra.run.smithi190:node-exporter.c> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@node-exporter.c.service 2024-02-07T13:34:12.359 INFO:tasks.cephadm:Adding alertmanager.a on smithi094 2024-02-07T13:34:12.359 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch apply alertmanager '1;smithi094=a' 2024-02-07T13:34:12.632 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:34:13.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:12 smithi094 ceph-mon[25576]: Checking dashboard <-> RGW credentials 2024-02-07T13:34:13.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:12 smithi094 ceph-mon[25576]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi094=a;smithi138=b;smithi190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:13.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:12 smithi094 ceph-mon[25576]: Saving service node-exporter spec with placement smithi094=a;smithi138=b;smithi190=c;count:3 2024-02-07T13:34:13.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:12 smithi094 ceph-mon[25576]: osdmap e24: 3 total, 3 up, 3 in 2024-02-07T13:34:13.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:12 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:34:13.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:13 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:34:13.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:13 smithi094 ceph-mon[25576]: pgmap v94: 65 pgs: 64 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:13.264 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:13 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:13.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:13 smithi190 ceph-mon[30713]: Checking dashboard <-> RGW credentials 2024-02-07T13:34:13.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:13 smithi190 ceph-mon[30713]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi094=a;smithi138=b;smithi190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:13.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:13 smithi190 ceph-mon[30713]: Saving service node-exporter spec with placement smithi094=a;smithi138=b;smithi190=c;count:3 2024-02-07T13:34:13.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:13 smithi190 ceph-mon[30713]: osdmap e24: 3 total, 3 up, 3 in 2024-02-07T13:34:13.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:13 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:34:13.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:13 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:34:13.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:13 smithi190 ceph-mon[30713]: pgmap v94: 65 pgs: 64 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:13.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:13 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:13.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:13 smithi138 ceph-mon[30535]: Checking dashboard <-> RGW credentials 2024-02-07T13:34:13.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:13 smithi138 ceph-mon[30535]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi094=a;smithi138=b;smithi190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:13.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:13 smithi138 ceph-mon[30535]: Saving service node-exporter spec with placement smithi094=a;smithi138=b;smithi190=c;count:3 2024-02-07T13:34:13.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:13 smithi138 ceph-mon[30535]: osdmap e24: 3 total, 3 up, 3 in 2024-02-07T13:34:13.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:13 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:34:13.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:13 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:34:13.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:13 smithi138 ceph-mon[30535]: pgmap v94: 65 pgs: 64 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:13.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:13 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:14.491 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled alertmanager update... 2024-02-07T13:34:14.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:14 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:34:14.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:14 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:34:14.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:14 smithi190 ceph-mon[30713]: osdmap e25: 3 total, 3 up, 3 in 2024-02-07T13:34:14.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:14 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:34:14.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:14 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:34:14.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:14 smithi094 ceph-mon[25576]: osdmap e25: 3 total, 3 up, 3 in 2024-02-07T13:34:14.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:14 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:34:14.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:14 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:34:14.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:14 smithi138 ceph-mon[30535]: osdmap e25: 3 total, 3 up, 3 in 2024-02-07T13:34:15.120 DEBUG:teuthology.orchestra.run.smithi094:alertmanager.a> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@alertmanager.a.service 2024-02-07T13:34:15.123 INFO:tasks.cephadm:Adding grafana.a on smithi138 2024-02-07T13:34:15.124 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph orch apply grafana '1;smithi138=a' 2024-02-07T13:34:15.358 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:34:15.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:15 smithi094 ceph-mon[25576]: osdmap e26: 3 total, 3 up, 3 in 2024-02-07T13:34:15.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:15 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:34:15.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:15 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:34:15.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:15 smithi094 ceph-mon[25576]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi094=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:15.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:15 smithi094 ceph-mon[25576]: Saving service alertmanager spec with placement smithi094=a;count:1 2024-02-07T13:34:15.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:15 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:15.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:15 smithi094 ceph-mon[25576]: pgmap v97: 97 pgs: 73 unknown, 24 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-07T13:34:15.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:15 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:34:15.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:15 smithi190 ceph-mon[30713]: osdmap e26: 3 total, 3 up, 3 in 2024-02-07T13:34:15.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:15 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:34:15.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:15 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:34:15.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:15 smithi190 ceph-mon[30713]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi094=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:15.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:15 smithi190 ceph-mon[30713]: Saving service alertmanager spec with placement smithi094=a;count:1 2024-02-07T13:34:15.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:15 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:15.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:15 smithi190 ceph-mon[30713]: pgmap v97: 97 pgs: 73 unknown, 24 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-07T13:34:15.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:15 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:34:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:15 smithi138 ceph-mon[30535]: osdmap e26: 3 total, 3 up, 3 in 2024-02-07T13:34:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:15 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:34:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:15 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:34:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:15 smithi138 ceph-mon[30535]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi094=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:15 smithi138 ceph-mon[30535]: Saving service alertmanager spec with placement smithi094=a;count:1 2024-02-07T13:34:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:15 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:15 smithi138 ceph-mon[30535]: pgmap v97: 97 pgs: 73 unknown, 24 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-07T13:34:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:15 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:34:16.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:16 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:34:16.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:16 smithi190 ceph-mon[30713]: osdmap e27: 3 total, 3 up, 3 in 2024-02-07T13:34:16.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:16 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:34:16.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:16 smithi094 ceph-mon[25576]: osdmap e27: 3 total, 3 up, 3 in 2024-02-07T13:34:16.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:16 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:34:16.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:16 smithi138 ceph-mon[30535]: osdmap e27: 3 total, 3 up, 3 in 2024-02-07T13:34:17.235 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled grafana update... 2024-02-07T13:34:17.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:17 smithi190 ceph-mon[30713]: osdmap e28: 3 total, 3 up, 3 in 2024-02-07T13:34:17.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:17 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:34:17.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:17 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:34:17.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:17 smithi190 ceph-mon[30713]: pgmap v100: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-07T13:34:17.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:17 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:17.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:17 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:34:17.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:17 smithi094 ceph-mon[25576]: osdmap e28: 3 total, 3 up, 3 in 2024-02-07T13:34:17.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:17 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:34:17.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:17 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:34:17.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:17 smithi094 ceph-mon[25576]: pgmap v100: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-07T13:34:17.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:17 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:17.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:17 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:34:17.766 DEBUG:teuthology.orchestra.run.smithi138:grafana.a> sudo journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@grafana.a.service 2024-02-07T13:34:17.769 INFO:tasks.cephadm:Setting up client nodes... 2024-02-07T13:34:17.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-07T13:34:17.794 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:17 smithi138 ceph-mon[30535]: osdmap e28: 3 total, 3 up, 3 in 2024-02-07T13:34:17.794 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:17 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:34:17.794 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:17 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:34:17.794 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:17 smithi138 ceph-mon[30535]: pgmap v100: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-07T13:34:17.794 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:17 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:17.794 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:17 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:34:18.012 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:18.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:18 smithi094 ceph-mon[25576]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi138=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:18.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:18 smithi094 ceph-mon[25576]: Saving service grafana spec with placement smithi138=a;count:1 2024-02-07T13:34:18.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:18 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:34:18.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:18 smithi094 ceph-mon[25576]: osdmap e29: 3 total, 3 up, 3 in 2024-02-07T13:34:18.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:18 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:34:18.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:18 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:34:18.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:18 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:34:18.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:18 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:34:18.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:18 smithi094 ceph-mon[25576]: osdmap e30: 3 total, 3 up, 3 in 2024-02-07T13:34:18.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:18 smithi190 ceph-mon[30713]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi138=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:18.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:18 smithi190 ceph-mon[30713]: Saving service grafana spec with placement smithi138=a;count:1 2024-02-07T13:34:18.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:18 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:34:18.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:18 smithi190 ceph-mon[30713]: osdmap e29: 3 total, 3 up, 3 in 2024-02-07T13:34:18.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:18 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:34:18.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:18 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:34:18.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:18 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:34:18.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:18 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:34:18.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:18 smithi190 ceph-mon[30713]: osdmap e30: 3 total, 3 up, 3 in 2024-02-07T13:34:18.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:18 smithi138 ceph-mon[30535]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi138=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:34:18.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:18 smithi138 ceph-mon[30535]: Saving service grafana spec with placement smithi138=a;count:1 2024-02-07T13:34:18.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:18 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:34:18.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:18 smithi138 ceph-mon[30535]: osdmap e29: 3 total, 3 up, 3 in 2024-02-07T13:34:18.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:18 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:34:18.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:18 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:34:18.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:18 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3768944072' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:34:18.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:18 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/1298163201' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:34:18.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:18 smithi138 ceph-mon[30535]: osdmap e30: 3 total, 3 up, 3 in 2024-02-07T13:34:19.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:19 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:19.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:19 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:19.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:19 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:19.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:19 smithi094 ceph-mon[25576]: pgmap v103: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 767 B/s wr, 1 op/s 2024-02-07T13:34:19.916 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-02-07T13:34:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDbhsNl+1qSNhAAmMW97IzjaftZRzDwBekjvw== 2024-02-07T13:34:20.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:19 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:20.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:19 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:20.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:19 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:20.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:19 smithi190 ceph-mon[30713]: pgmap v103: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 767 B/s wr, 1 op/s 2024-02-07T13:34:20.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:19 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:20.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:19 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:20.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:19 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:20.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:19 smithi138 ceph-mon[30535]: pgmap v103: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 767 B/s wr, 1 op/s 2024-02-07T13:34:20.519 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:34:20.519 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-07T13:34:20.519 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-07T13:34:20.562 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-07T13:34:20.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:20 smithi094 ceph-mon[25576]: Deploying daemon prometheus.a on smithi138 2024-02-07T13:34:20.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:20 smithi094 ceph-mon[25576]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T13:34:20.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:20 smithi094 ceph-mon[25576]: Cluster is now healthy 2024-02-07T13:34:20.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:20 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2282787079' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:20.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:20 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2282787079' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:34:20.827 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.b/config 2024-02-07T13:34:20.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:20 smithi138 ceph-mon[30535]: Deploying daemon prometheus.a on smithi138 2024-02-07T13:34:20.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:20 smithi138 ceph-mon[30535]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T13:34:20.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:20 smithi138 ceph-mon[30535]: Cluster is now healthy 2024-02-07T13:34:20.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:20 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/2282787079' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:20.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:20 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/2282787079' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:34:21.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:20 smithi190 ceph-mon[30713]: Deploying daemon prometheus.a on smithi138 2024-02-07T13:34:21.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:20 smithi190 ceph-mon[30713]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T13:34:21.031 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:20 smithi190 ceph-mon[30713]: Cluster is now healthy 2024-02-07T13:34:21.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:20 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/2282787079' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:21.032 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:20 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/2282787079' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:34:21.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:21 smithi138 ceph-mon[30535]: pgmap v104: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 89 KiB/s rd, 6.9 KiB/s wr, 215 op/s 2024-02-07T13:34:22.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:21 smithi094 ceph-mon[25576]: pgmap v104: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 89 KiB/s rd, 6.9 KiB/s wr, 215 op/s 2024-02-07T13:34:22.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:21 smithi190 ceph-mon[30713]: pgmap v104: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 89 KiB/s rd, 6.9 KiB/s wr, 215 op/s 2024-02-07T13:34:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-02-07T13:34:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDfhsNlMQDFGxAAyVliikSvvUyyiujHueFBcw== 2024-02-07T13:34:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:23 smithi094 ceph-mon[25576]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 75 KiB/s rd, 5.8 KiB/s wr, 180 op/s 2024-02-07T13:34:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:23 smithi094 ceph-mon[25576]: from='client.? 172.21.15.138:0/710933238' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:23 smithi094 ceph-mon[25576]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:24.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:23 smithi094 ceph-mon[25576]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:34:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:23 smithi190 ceph-mon[30713]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 75 KiB/s rd, 5.8 KiB/s wr, 180 op/s 2024-02-07T13:34:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:23 smithi190 ceph-mon[30713]: from='client.? 172.21.15.138:0/710933238' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:23 smithi190 ceph-mon[30713]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:24.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:23 smithi190 ceph-mon[30713]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:34:24.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:23 smithi138 ceph-mon[30535]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 75 KiB/s rd, 5.8 KiB/s wr, 180 op/s 2024-02-07T13:34:24.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:23 smithi138 ceph-mon[30535]: from='client.? 172.21.15.138:0/710933238' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:24.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:23 smithi138 ceph-mon[30535]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:24.095 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:23 smithi138 ceph-mon[30535]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:34:25.554 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:34:25.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-07T13:34:25.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-07T13:34:25.599 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-07T13:34:25.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:25 smithi138 ceph-mon[30535]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-07T13:34:25.846 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:34:25.967 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:25 smithi190 ceph-mon[30713]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-07T13:34:26.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:25 smithi094 ceph-mon[25576]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-07T13:34:27.714 INFO:teuthology.orchestra.run.smithi190.stdout:[client.2] 2024-02-07T13:34:27.714 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDjhsNlIIFSKhAArdFST+opDTgHiKrEtepWmA== 2024-02-07T13:34:27.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:27 smithi138 ceph-mon[30535]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 50 KiB/s rd, 3.9 KiB/s wr, 122 op/s 2024-02-07T13:34:27.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:27 smithi138 ceph-mon[30535]: from='client.? 172.21.15.190:0/258868246' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:27.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:27 smithi138 ceph-mon[30535]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:27.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:27 smithi138 ceph-mon[30535]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:34:28.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:27 smithi094 ceph-mon[25576]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 50 KiB/s rd, 3.9 KiB/s wr, 122 op/s 2024-02-07T13:34:28.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:27 smithi094 ceph-mon[25576]: from='client.? 172.21.15.190:0/258868246' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:28.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:27 smithi094 ceph-mon[25576]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:28.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:27 smithi094 ceph-mon[25576]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:34:28.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:27 smithi190 ceph-mon[30713]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 50 KiB/s rd, 3.9 KiB/s wr, 122 op/s 2024-02-07T13:34:28.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:27 smithi190 ceph-mon[30713]: from='client.? 172.21.15.190:0/258868246' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:28.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:27 smithi190 ceph-mon[30713]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:34:28.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:27 smithi190 ceph-mon[30713]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:34:28.345 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:34:28.345 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-02-07T13:34:28.345 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-07T13:34:28.386 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-07T13:34:28.386 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-07T13:34:28.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph mgr dump --format=json 2024-02-07T13:34:28.630 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:29.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:29 smithi094 ceph-mon[25576]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 45 KiB/s rd, 3.5 KiB/s wr, 110 op/s 2024-02-07T13:34:29.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:29 smithi190 ceph-mon[30713]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 45 KiB/s rd, 3.5 KiB/s wr, 110 op/s 2024-02-07T13:34:29.805 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:29 smithi138 ceph-mon[30535]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 45 KiB/s rd, 3.5 KiB/s wr, 110 op/s 2024-02-07T13:34:29.806 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:29 smithi138 systemd[1]: Starting Ceph prometheus.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:34:30.061 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:29 smithi138 podman[40925]: 2024-02-07 13:34:29.805962191 +0000 UTC m=+0.104387146 container create 98a954e2991d46265567f51ed6a1bf465b67ccfcb1828a194f4ebf7d8e7792dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:34:30.061 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:29 smithi138 podman[40925]: 2024-02-07 13:34:29.720922085 +0000 UTC m=+0.019347047 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-07T13:34:30.061 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 podman[40925]: 2024-02-07 13:34:30.03326183 +0000 UTC m=+0.331686785 container init 98a954e2991d46265567f51ed6a1bf465b67ccfcb1828a194f4ebf7d8e7792dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:34:30.061 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 podman[40925]: 2024-02-07 13:34:30.038451102 +0000 UTC m=+0.336876058 container start 98a954e2991d46265567f51ed6a1bf465b67ccfcb1828a194f4ebf7d8e7792dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:34:30.343 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.061Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-07T13:34:30.343 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.062Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-07T13:34:30.343 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.062Z caller=main.go:570 level=info host_details="(Linux 5.14.0-418.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Feb 5 23:17:48 UTC 2024 x86_64 smithi138 (none))" 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.062Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.062Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.063Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.063Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.065Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.065Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.066Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.067Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.863µs 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.067Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.067Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.067Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.524µs wal_replay_duration=196.385µs wbl_replay_duration=104ns total_replay_duration=230.864µs 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.067Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.067Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-07T13:34:30.344 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.067Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-07T13:34:30.345 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.085Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=17.231685ms db_storage=718ns remote_storage=1.023µs web_handler=301ns query_engine=588ns scrape=397.305µs scrape_sd=64.215µs notify=891ns notify_sd=2.084µs rules=16.592374ms tracing=6.294µs 2024-02-07T13:34:30.345 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.085Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-07T13:34:30.345 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:34:30.085Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-07T13:34:30.345 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 bash[40925]: 98a954e2991d46265567f51ed6a1bf465b67ccfcb1828a194f4ebf7d8e7792dc 2024-02-07T13:34:30.345 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:34:30 smithi138 systemd[1]: Started Ceph prometheus.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:34:30.681 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:34:31.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:31.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:31.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:31.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-07T13:34:31.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-mon[25576]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 39 KiB/s rd, 3.0 KiB/s wr, 93 op/s 2024-02-07T13:34:31.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3985122055' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-07T13:34:31.369 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":540969863},{"type":"v1","addr":"172.21.15.94:6801","nonce":540969863}]},"active_addr":"172.21.15.94:6801/540969863","active_change":"2024-02-07T13:31:32.635948+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"x","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.94:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1271527060}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3890251074}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3996659415}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2160770702}]}]} 2024-02-07T13:34:31.375 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-07T13:34:31.375 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-07T13:34:31.375 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-07T13:34:31.531 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: ignoring --setuser ceph since I am not root 2024-02-07T13:34:31.531 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: ignoring --setgroup ceph since I am not root 2024-02-07T13:34:31.531 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-mgr[32814]: -- 172.21.15.190:0/654891988 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55acff5df520 con 0x55acff5b4c00 2024-02-07T13:34:31.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:31.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:31.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:31.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-07T13:34:31.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-mon[30713]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 39 KiB/s rd, 3.0 KiB/s wr, 93 op/s 2024-02-07T13:34:31.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3985122055' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-07T13:34:31.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:31.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:31.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' 2024-02-07T13:34:31.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-07T13:34:31.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-mon[30535]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 39 KiB/s rd, 3.0 KiB/s wr, 93 op/s 2024-02-07T13:34:31.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3985122055' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-07T13:34:31.594 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: ignoring --setuser ceph since I am not root 2024-02-07T13:34:31.594 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: ignoring --setgroup ceph since I am not root 2024-02-07T13:34:31.594 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-mgr[32014]: -- 172.21.15.138:0/2645824369 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5654c1be1520 con 0x5654c1bb6c00 2024-02-07T13:34:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:31.748 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ignoring --setuser ceph since I am not root 2024-02-07T13:34:31.748 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ignoring --setgroup ceph since I am not root 2024-02-07T13:34:32.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:31.855+0000 7fed016ae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:34:32.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:31 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:31.954+0000 7fed016ae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:34:32.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:31.826+0000 7ff358ee1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:34:32.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:31 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:31.917+0000 7ff358ee1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:34:32.260 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:31.827+0000 7f18d896b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:34:32.260 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:31 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:31.919+0000 7f18d896b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:34:32.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:32 smithi094 ceph-mon[25576]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-07T13:34:32.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:32 smithi094 ceph-mon[25576]: mgrmap e17: y(active, since 2m), standbys: z, x 2024-02-07T13:34:32.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:32 smithi190 ceph-mon[30713]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-07T13:34:32.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:32 smithi190 ceph-mon[30713]: mgrmap e17: y(active, since 2m), standbys: z, x 2024-02-07T13:34:32.548 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:32 smithi138 ceph-mon[30535]: from='mgr.14152 172.21.15.94:0/4095209137' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-07T13:34:32.548 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:32 smithi138 ceph-mon[30535]: mgrmap e17: y(active, since 2m), standbys: z, x 2024-02-07T13:34:32.843 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:32 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:32.548+0000 7ff358ee1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:34:32.843 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:32 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:32.640+0000 7ff358ee1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:34:32.843 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:32 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:32.779+0000 7ff358ee1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:34:32.878 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:32 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:32.550+0000 7f18d896b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:34:32.878 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:32 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:32.642+0000 7f18d896b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:34:32.878 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:32 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:32.784+0000 7f18d896b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:34:32.878 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:32 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:32.878+0000 7f18d896b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:34:32.890 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:32 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:32.599+0000 7fed016ae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:34:32.890 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:32 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:32.693+0000 7fed016ae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:34:32.890 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:32 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:32.835+0000 7fed016ae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:34:33.176 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:32 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:32.932+0000 7fed016ae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:34:33.176 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:33 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:33.019+0000 7fed016ae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:34:33.218 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:32 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:32.873+0000 7ff358ee1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:34:33.218 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:32 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:32.955+0000 7ff358ee1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:34:33.219 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:33 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:33.218+0000 7ff358ee1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:34:33.220 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:32 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:32.960+0000 7f18d896b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:34:33.220 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:33 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:33.220+0000 7f18d896b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:34:33.435 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:33 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:33.287+0000 7fed016ae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:34:33.435 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:33 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:33.400+0000 7fed016ae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:34:33.510 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:33 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:33.330+0000 7f18d896b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:34:33.510 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:33 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:33.416+0000 7f18d896b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:34:33.510 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:33 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:33.510+0000 7f18d896b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:34:33.511 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:33 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:33.327+0000 7ff358ee1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:34:33.511 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:33 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:33.414+0000 7ff358ee1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:34:33.511 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:33 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:33.510+0000 7ff358ee1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:34:33.641 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:34:33.642 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":30,"fsid":"edf33ada-c5bc-11ee-95b6-87774f69a715","created":"2024-02-07T13:30:33.379810+0000","modified":"2024-02-07T13:34:18.433598+0000","last_up_change":"2024-02-07T13:33:58.013510+0000","last_in_change":"2024-02-07T13:33:39.700912+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-07T13:33:58.966796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-07T13:34:09.773884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-07T13:34:11.715875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-07T13:34:13.666728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-07T13:34:15.726170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a54e30c-a30a-4824-be90-1326fb7d73fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6803","nonce":3542023933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6805","nonce":3542023933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6809","nonce":3542023933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6807","nonce":3542023933}]},"public_addr":"172.21.15.94:6803/3542023933","cluster_addr":"172.21.15.94:6805/3542023933","heartbeat_back_addr":"172.21.15.94:6809/3542023933","heartbeat_front_addr":"172.21.15.94:6807/3542023933","state":["exists","up"]},{"osd":1,"uuid":"95884813-942c-4343-9aa4-ed974b154348","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6801","nonce":372706022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6803","nonce":372706022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6807","nonce":372706022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6805","nonce":372706022}]},"public_addr":"172.21.15.138:6801/372706022","cluster_addr":"172.21.15.138:6803/372706022","heartbeat_back_addr":"172.21.15.138:6807/372706022","heartbeat_front_addr":"172.21.15.138:6805/372706022","state":["exists","up"]},{"osd":2,"uuid":"770fe04a-021a-49a9-b88b-5f4ab50822b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6801","nonce":2499326806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6803","nonce":2499326806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6807","nonce":2499326806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6805","nonce":2499326806}]},"public_addr":"172.21.15.190:6801/2499326806","cluster_addr":"172.21.15.190:6803/2499326806","heartbeat_back_addr":"172.21.15.190:6807/2499326806","heartbeat_front_addr":"172.21.15.190:6805/2499326806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:33:07.338337+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:33:32.050787+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:33:56.163828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/3836439501":"2024-02-08T13:31:32.635798+0000","172.21.15.94:6800/3836439501":"2024-02-08T13:31:32.635798+0000","172.21.15.94:0/2055038774":"2024-02-08T13:31:32.635798+0000","172.21.15.94:6801/2086851078":"2024-02-08T13:31:06.303667+0000","172.21.15.94:6800/2086851078":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/3678689078":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/1966097319":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/1166789205":"2024-02-08T13:31:32.635798+0000","172.21.15.94:0/43997712":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/2217401373":"2024-02-08T13:31:32.635798+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-07T13:34:33.763 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:33 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:33.491+0000 7fed016ae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:34:33.763 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:33 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:33.588+0000 7fed016ae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:34:33.780 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:33 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:33.685+0000 7f18d896b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:34:33.843 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:33 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:33.686+0000 7ff358ee1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:34:34.256 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:33 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:33.768+0000 7fed016ae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:34:34.285 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-07T13:34:34.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-07T13:34:34.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:34 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2751492642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:34:34.544 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:34.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:34 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/2751492642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:34:34.639 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:34 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/2751492642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:34:34.918 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:34 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:34.760+0000 7fed016ae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:34:34.918 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:34 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:34.897+0000 7fed016ae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:34:35.030 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:34 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:34.637+0000 7f18d896b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:34:35.030 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:34 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:34.768+0000 7f18d896b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:34:35.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:34 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:34.639+0000 7ff358ee1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:34:35.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:34 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:34.768+0000 7ff358ee1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:34:35.383 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:35 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:35.099+0000 7ff358ee1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:34:35.383 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:35 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:35.188+0000 7ff358ee1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:34:35.383 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:35 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:35.382+0000 7ff358ee1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:34:35.388 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:35 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:35.102+0000 7f18d896b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:34:35.388 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:35 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:35.192+0000 7f18d896b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:34:35.388 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:35 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:35.388+0000 7f18d896b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:34:35.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:35 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:35.250+0000 7fed016ae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:34:35.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:35 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:35.344+0000 7fed016ae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:34:35.780 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:35 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:35.573+0000 7f18d896b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:34:35.838 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:35 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:35.542+0000 7fed016ae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:34:35.838 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:35 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:35.732+0000 7fed016ae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:34:35.842 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:35 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:35.567+0000 7ff358ee1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:34:36.111 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:35 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:35.849+0000 7ff358ee1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:34:36.111 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:36 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:36.111+0000 7ff358ee1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:34:36.115 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:35 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:35.855+0000 7f18d896b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:34:36.115 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:36 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:36.115+0000 7f18d896b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:34:36.184 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:36 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:36.023+0000 7fed016ae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:34:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:34:36.496 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":30,"fsid":"edf33ada-c5bc-11ee-95b6-87774f69a715","created":"2024-02-07T13:30:33.379810+0000","modified":"2024-02-07T13:34:18.433598+0000","last_up_change":"2024-02-07T13:33:58.013510+0000","last_in_change":"2024-02-07T13:33:39.700912+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-07T13:33:58.966796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-07T13:34:09.773884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-07T13:34:11.715875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-07T13:34:13.666728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-07T13:34:15.726170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a54e30c-a30a-4824-be90-1326fb7d73fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6803","nonce":3542023933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6805","nonce":3542023933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6809","nonce":3542023933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3542023933},{"type":"v1","addr":"172.21.15.94:6807","nonce":3542023933}]},"public_addr":"172.21.15.94:6803/3542023933","cluster_addr":"172.21.15.94:6805/3542023933","heartbeat_back_addr":"172.21.15.94:6809/3542023933","heartbeat_front_addr":"172.21.15.94:6807/3542023933","state":["exists","up"]},{"osd":1,"uuid":"95884813-942c-4343-9aa4-ed974b154348","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6801","nonce":372706022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6803","nonce":372706022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6807","nonce":372706022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":372706022},{"type":"v1","addr":"172.21.15.138:6805","nonce":372706022}]},"public_addr":"172.21.15.138:6801/372706022","cluster_addr":"172.21.15.138:6803/372706022","heartbeat_back_addr":"172.21.15.138:6807/372706022","heartbeat_front_addr":"172.21.15.138:6805/372706022","state":["exists","up"]},{"osd":2,"uuid":"770fe04a-021a-49a9-b88b-5f4ab50822b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6801","nonce":2499326806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6803","nonce":2499326806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6807","nonce":2499326806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2499326806},{"type":"v1","addr":"172.21.15.190:6805","nonce":2499326806}]},"public_addr":"172.21.15.190:6801/2499326806","cluster_addr":"172.21.15.190:6803/2499326806","heartbeat_back_addr":"172.21.15.190:6807/2499326806","heartbeat_front_addr":"172.21.15.190:6805/2499326806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:33:07.338337+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:33:32.050787+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:33:56.163828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/3836439501":"2024-02-08T13:31:32.635798+0000","172.21.15.94:6800/3836439501":"2024-02-08T13:31:32.635798+0000","172.21.15.94:0/2055038774":"2024-02-08T13:31:32.635798+0000","172.21.15.94:6801/2086851078":"2024-02-08T13:31:06.303667+0000","172.21.15.94:6800/2086851078":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/3678689078":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/1966097319":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/1166789205":"2024-02-08T13:31:32.635798+0000","172.21.15.94:0/43997712":"2024-02-08T13:31:06.303667+0000","172.21.15.94:0/2217401373":"2024-02-08T13:31:32.635798+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-07T13:34:36.497 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:36 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:36.292+0000 7fed016ae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:34:36.530 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:36 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:36.351+0000 7f18d896b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:34:36.547 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:36 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:36.345+0000 7ff358ee1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:34:36.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:36 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/1964952604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:34:36.763 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:36 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:36.541+0000 7fed016ae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:34:36.842 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:36 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/1964952604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:34:37.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:36 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/1964952604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:34:37.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-07T13:34:37.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-07T13:34:37.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-07T13:34:37.376 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:37.380 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:37.410 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:37 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:37.409+0000 7fed016ae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:34:37.530 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:37 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:37.146+0000 7f18d896b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:34:37.593 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:37 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:37.143+0000 7ff358ee1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:34:38.022 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:37 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:37.711+0000 7ff358ee1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:34:38.023 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:37 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:37.809+0000 7ff358ee1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:34:38.023 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:38.022+0000 7ff358ee1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:34:38.026 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:37 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:37.714+0000 7f18d896b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:34:38.026 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:37 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:37.813+0000 7f18d896b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:34:38.026 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:38.026+0000 7f18d896b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:34:38.069 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:38.002+0000 7fed016ae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:34:38.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: Standby manager daemon x restarted 2024-02-07T13:34:38.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: Standby manager daemon x started 2024-02-07T13:34:38.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: Standby manager daemon z restarted 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: Standby manager daemon z started 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-mon[30713]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: [07/Feb/2024:13:34:38] ENGINE Bus STARTING 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: CherryPy Checker: 2024-02-07T13:34:38.281 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: The Application mounted at '' has an empty config. 2024-02-07T13:34:38.282 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: 2024-02-07T13:34:38.282 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: [07/Feb/2024:13:34:38] ENGINE Serving on http://:::9283 2024-02-07T13:34:38.282 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:34:38 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z[32790]: [07/Feb/2024:13:34:38] ENGINE Bus STARTED 2024-02-07T13:34:38.331 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:38.111+0000 7fed016ae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:34:38.332 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:38.331+0000 7fed016ae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:34:38.332 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: Standby manager daemon x restarted 2024-02-07T13:34:38.332 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: Standby manager daemon x started 2024-02-07T13:34:38.332 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:34:38.332 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:34:38.332 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: Standby manager daemon z restarted 2024-02-07T13:34:38.332 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: Standby manager daemon z started 2024-02-07T13:34:38.332 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:34:38.332 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:34:38.333 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:34:38.333 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:34:38.333 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:34:38.333 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-mon[25576]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:34:38.343 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: [07/Feb/2024:13:34:38] ENGINE Bus STARTING 2024-02-07T13:34:38.343 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: CherryPy Checker: 2024-02-07T13:34:38.343 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: The Application mounted at '' has an empty config. 2024-02-07T13:34:38.343 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: 2024-02-07T13:34:38.343 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: [07/Feb/2024:13:34:38] ENGINE Serving on http://:::9283 2024-02-07T13:34:38.343 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x[31990]: [07/Feb/2024:13:34:38] ENGINE Bus STARTED 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: Standby manager daemon x restarted 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: Standby manager daemon x started 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: Standby manager daemon z restarted 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: Standby manager daemon z started 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.138:0/3168903444' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:34:38.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:38 smithi138 ceph-mon[30535]: from='mgr.? 172.21.15.190:0/3807094762' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:34:38.590 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:34:38] ENGINE Bus STARTING 2024-02-07T13:34:38.590 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: CherryPy Checker: 2024-02-07T13:34:38.591 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: The Application mounted at '' has an empty config. 2024-02-07T13:34:38.591 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:34:38.591 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:34:38] ENGINE Serving on http://:::9283 2024-02-07T13:34:38.591 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:38 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:34:38] ENGINE Bus STARTED 2024-02-07T13:34:39.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-07T13:34:39.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: Active manager daemon y restarted 2024-02-07T13:34:39.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: Activating manager daemon y 2024-02-07T13:34:39.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: osdmap e31: 3 total, 3 up, 3 in 2024-02-07T13:34:39.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: mgrmap e19: y(active, starting, since 0.00951654s), standbys: x, z 2024-02-07T13:34:39.498 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: Manager daemon y is now available 2024-02-07T13:34:39.499 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:39.500 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:39.500 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:39.500 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:34:39.500 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:34:39.500 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:39 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:34:39.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-07T13:34:39.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: Active manager daemon y restarted 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: Activating manager daemon y 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: osdmap e31: 3 total, 3 up, 3 in 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: mgrmap e19: y(active, starting, since 0.00951654s), standbys: x, z 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:34:39.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:34:39.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:34:39.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: Manager daemon y is now available 2024-02-07T13:34:39.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:39.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:39.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:39.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:34:39.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:34:39.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:39 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:34:39.553 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-07T13:34:39.553 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: Active manager daemon y restarted 2024-02-07T13:34:39.553 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: Activating manager daemon y 2024-02-07T13:34:39.553 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: osdmap e31: 3 total, 3 up, 3 in 2024-02-07T13:34:39.553 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: mgrmap e19: y(active, starting, since 0.00951654s), standbys: x, z 2024-02-07T13:34:39.553 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:34:39.553 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: Manager daemon y is now available 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:39.554 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:34:39.555 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:39.555 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:34:39.555 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:34:39.555 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:39 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:34:40.398 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:40 smithi138 ceph-mon[30535]: [07/Feb/2024:13:34:38] ENGINE Bus STARTING 2024-02-07T13:34:40.399 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:40 smithi138 ceph-mon[30535]: [07/Feb/2024:13:34:38] ENGINE Serving on https://172.21.15.94:7150 2024-02-07T13:34:40.399 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:40 smithi138 ceph-mon[30535]: [07/Feb/2024:13:34:39] ENGINE Serving on http://172.21.15.94:8765 2024-02-07T13:34:40.399 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:40 smithi138 ceph-mon[30535]: [07/Feb/2024:13:34:39] ENGINE Bus STARTED 2024-02-07T13:34:40.399 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:40 smithi138 ceph-mon[30535]: mgrmap e20: y(active, since 1.0246s), standbys: x, z 2024-02-07T13:34:40.399 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:40 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:40.399 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:40 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:40.447 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:40 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ::ffff:172.21.15.138 - - [07/Feb/2024:13:34:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:34:40.448 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:40 smithi094 ceph-mon[25576]: [07/Feb/2024:13:34:38] ENGINE Bus STARTING 2024-02-07T13:34:40.448 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:40 smithi094 ceph-mon[25576]: [07/Feb/2024:13:34:38] ENGINE Serving on https://172.21.15.94:7150 2024-02-07T13:34:40.448 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:40 smithi094 ceph-mon[25576]: [07/Feb/2024:13:34:39] ENGINE Serving on http://172.21.15.94:8765 2024-02-07T13:34:40.448 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:40 smithi094 ceph-mon[25576]: [07/Feb/2024:13:34:39] ENGINE Bus STARTED 2024-02-07T13:34:40.448 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:40 smithi094 ceph-mon[25576]: mgrmap e20: y(active, since 1.0246s), standbys: x, z 2024-02-07T13:34:40.448 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:40 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:40.448 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:40 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:40.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:40 smithi190 ceph-mon[30713]: [07/Feb/2024:13:34:38] ENGINE Bus STARTING 2024-02-07T13:34:40.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:40 smithi190 ceph-mon[30713]: [07/Feb/2024:13:34:38] ENGINE Serving on https://172.21.15.94:7150 2024-02-07T13:34:40.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:40 smithi190 ceph-mon[30713]: [07/Feb/2024:13:34:39] ENGINE Serving on http://172.21.15.94:8765 2024-02-07T13:34:40.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:40 smithi190 ceph-mon[30713]: [07/Feb/2024:13:34:39] ENGINE Bus STARTED 2024-02-07T13:34:40.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:40 smithi190 ceph-mon[30713]: mgrmap e20: y(active, since 1.0246s), standbys: x, z 2024-02-07T13:34:40.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:40 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:40.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:40 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:41 smithi094 ceph-mon[25576]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:41.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:41 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:41 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:41 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:41 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:41 smithi190 ceph-mon[30713]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:41.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:41 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:41 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:41 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:41 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.546 INFO:teuthology.orchestra.run.smithi094.stdout:55834574864 2024-02-07T13:34:41.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-07T13:34:41.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:41 smithi138 ceph-mon[30535]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:41.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:41 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:41 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:41 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:41.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:41 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:42 smithi094 ceph-mon[25576]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:42.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:42 smithi094 ceph-mon[25576]: mgrmap e21: y(active, since 2s), standbys: x, z 2024-02-07T13:34:42.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:42 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:42 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:42 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:34:42.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:42 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:42 smithi190 ceph-mon[30713]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:42.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:42 smithi190 ceph-mon[30713]: mgrmap e21: y(active, since 2s), standbys: x, z 2024-02-07T13:34:42.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:42 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:42 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:42 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:34:42.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:42 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:42 smithi138 ceph-mon[30535]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:42.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:42 smithi138 ceph-mon[30535]: mgrmap e21: y(active, since 2s), standbys: x, z 2024-02-07T13:34:42.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:42 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:42 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:42 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:34:42.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:42 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:42.864 INFO:teuthology.orchestra.run.smithi094.stdout:34359738389 2024-02-07T13:34:42.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-07T13:34:43.431 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:43 smithi190 ceph-mon[30713]: Adjusting osd_memory_target on smithi138 to 16064M 2024-02-07T13:34:43.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:43 smithi094 ceph-mon[25576]: Adjusting osd_memory_target on smithi138 to 16064M 2024-02-07T13:34:43.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:43 smithi138 ceph-mon[30535]: Adjusting osd_memory_target on smithi138 to 16064M 2024-02-07T13:34:43.745 INFO:teuthology.orchestra.run.smithi094.stdout:77309411339 2024-02-07T13:34:43.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-07T13:34:43.760 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:44.033 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:44.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:44 smithi094 ceph-mon[25576]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:44.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:44 smithi094 ceph-mon[25576]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-07T13:34:44.448 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:44 smithi190 ceph-mon[30713]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:44.448 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:44 smithi190 ceph-mon[30713]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-07T13:34:44.494 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:44 smithi138 ceph-mon[30535]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:44.494 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:44 smithi138 ceph-mon[30535]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-07T13:34:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:46.321 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:46 smithi094 ceph-mon[25576]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:46.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:46 smithi190 ceph-mon[30713]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:46.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:46 smithi138 ceph-mon[30535]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:34:48.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:48 smithi094 ceph-mon[25576]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-07T13:34:48.318 INFO:teuthology.orchestra.run.smithi094.stdout:77309411340 2024-02-07T13:34:48.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:48 smithi190 ceph-mon[30713]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-07T13:34:48.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:48 smithi138 ceph-mon[30535]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-07T13:34:48.900 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411339 got 77309411340 for osd.2 2024-02-07T13:34:48.900 DEBUG:teuthology.parallel:result is None 2024-02-07T13:34:49.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:49 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2577406063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-07T13:34:49.330 INFO:teuthology.orchestra.run.smithi094.stdout:34359738390 2024-02-07T13:34:49.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:49 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/2577406063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-07T13:34:49.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:49 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/2577406063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-07T13:34:49.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738389 got 34359738390 for osd.0 2024-02-07T13:34:49.998 DEBUG:teuthology.parallel:result is None 2024-02-07T13:34:50.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:34:50 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ::ffff:172.21.15.138 - - [07/Feb/2024:13:34:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:34:50.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:50 smithi094 ceph-mon[25576]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-07T13:34:50.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:50 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:50.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:50 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:50.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:50 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/4235303132' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-07T13:34:50.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:50 smithi190 ceph-mon[30713]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-07T13:34:50.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:50 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:50.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:50 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:50.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:50 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/4235303132' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-07T13:34:50.564 INFO:teuthology.orchestra.run.smithi094.stdout:55834574865 2024-02-07T13:34:50.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:50 smithi138 ceph-mon[30535]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-07T13:34:50.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:50 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:50.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:50 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:50.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:50 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/4235303132' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-07T13:34:51.325 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574864 got 55834574865 for osd.1 2024-02-07T13:34:51.325 DEBUG:teuthology.parallel:result is None 2024-02-07T13:34:51.325 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-07T13:34:51.325 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-07T13:34:51.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:51 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/3240707082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-07T13:34:51.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:51 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:51.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:51 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:51.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:51 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:34:51.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:51 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:51.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:51 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:51.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:51 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:51.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:51 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/3240707082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-07T13:34:51.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:51 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:51.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:51 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:51.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:51 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:34:51.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:51 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:51.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:51 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:51.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:51 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:51.571 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:51.592 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:51 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/3240707082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-07T13:34:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:51 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:51 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:51 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:34:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:51 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:51 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:34:51.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:51 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:34:52.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-07T13:34:52.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Adjusting osd_memory_target on smithi094 to 16064M 2024-02-07T13:34:52.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-07T13:34:52.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Updating smithi138:/etc/ceph/ceph.conf 2024-02-07T13:34:52.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Updating smithi190:/etc/ceph/ceph.conf 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:52 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-07T13:34:52.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Adjusting osd_memory_target on smithi094 to 16064M 2024-02-07T13:34:52.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-07T13:34:52.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Updating smithi138:/etc/ceph/ceph.conf 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Updating smithi190:/etc/ceph/ceph.conf 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:52 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-07T13:34:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Adjusting osd_memory_target on smithi094 to 16064M 2024-02-07T13:34:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-07T13:34:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Updating smithi138:/etc/ceph/ceph.conf 2024-02-07T13:34:52.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Updating smithi190:/etc/ceph/ceph.conf 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:52.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:52 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:34:53.361 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:53 smithi094 ceph-mon[25576]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:34:53.361 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:53 smithi094 ceph-mon[25576]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:34:53.361 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:53 smithi094 ceph-mon[25576]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:34:53.361 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:53 smithi094 ceph-mon[25576]: Deploying daemon alertmanager.a on smithi094 2024-02-07T13:34:53.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:53 smithi190 ceph-mon[30713]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:34:53.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:53 smithi190 ceph-mon[30713]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:34:53.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:53 smithi190 ceph-mon[30713]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:34:53.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:53 smithi190 ceph-mon[30713]: Deploying daemon alertmanager.a on smithi094 2024-02-07T13:34:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:34:53.570 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-02-07T13:34:53.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:53 smithi138 ceph-mon[30535]: Updating smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:34:53.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:53 smithi138 ceph-mon[30535]: Updating smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:34:53.596 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:53 smithi138 ceph-mon[30535]: Updating smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:34:53.596 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:53 smithi138 ceph-mon[30535]: Deploying daemon alertmanager.a on smithi094 2024-02-07T13:34:54.216 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-02-07T13:34:52.362486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":100772,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":97659,"kb_avail":281122396,"statfs":{"total":287972524032,"available":287869333504,"internally_reserved":0,"allocated":3158016,"data_stored":2100177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":100003151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002717"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352694+0000","last_change":"2024-02-07T13:34:17.446711+0000","last_active":"2024-02-07T13:34:38.352694+0000","last_peered":"2024-02-07T13:34:38.352694+0000","last_clean":"2024-02-07T13:34:38.352694+0000","last_became_active":"2024-02-07T13:34:17.446312+0000","last_became_peered":"2024-02-07T13:34:17.446312+0000","last_unstale":"2024-02-07T13:34:38.352694+0000","last_undegraded":"2024-02-07T13:34:38.352694+0000","last_fullsized":"2024-02-07T13:34:38.352694+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:00:36.521357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354840+0000","last_change":"2024-02-07T13:34:15.432414+0000","last_active":"2024-02-07T13:34:38.354840+0000","last_peered":"2024-02-07T13:34:38.354840+0000","last_clean":"2024-02-07T13:34:38.354840+0000","last_became_active":"2024-02-07T13:34:15.432114+0000","last_became_peered":"2024-02-07T13:34:15.432114+0000","last_unstale":"2024-02-07T13:34:38.354840+0000","last_undegraded":"2024-02-07T13:34:38.354840+0000","last_fullsized":"2024-02-07T13:34:38.354840+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:42:13.628394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353543+0000","last_change":"2024-02-07T13:34:11.419834+0000","last_active":"2024-02-07T13:34:38.353543+0000","last_peered":"2024-02-07T13:34:38.353543+0000","last_clean":"2024-02-07T13:34:38.353543+0000","last_became_active":"2024-02-07T13:34:11.419488+0000","last_became_peered":"2024-02-07T13:34:11.419488+0000","last_unstale":"2024-02-07T13:34:38.353543+0000","last_undegraded":"2024-02-07T13:34:38.353543+0000","last_fullsized":"2024-02-07T13:34:38.353543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:29:52.258771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352836+0000","last_change":"2024-02-07T13:34:13.428893+0000","last_active":"2024-02-07T13:34:38.352836+0000","last_peered":"2024-02-07T13:34:38.352836+0000","last_clean":"2024-02-07T13:34:38.352836+0000","last_became_active":"2024-02-07T13:34:13.428518+0000","last_became_peered":"2024-02-07T13:34:13.428518+0000","last_unstale":"2024-02-07T13:34:38.352836+0000","last_undegraded":"2024-02-07T13:34:38.352836+0000","last_fullsized":"2024-02-07T13:34:38.352836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:20:01.615410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350135+0000","last_change":"2024-02-07T13:34:17.444471+0000","last_active":"2024-02-07T13:34:38.350135+0000","last_peered":"2024-02-07T13:34:38.350135+0000","last_clean":"2024-02-07T13:34:38.350135+0000","last_became_active":"2024-02-07T13:34:17.443947+0000","last_became_peered":"2024-02-07T13:34:17.443947+0000","last_unstale":"2024-02-07T13:34:38.350135+0000","last_undegraded":"2024-02-07T13:34:38.350135+0000","last_fullsized":"2024-02-07T13:34:38.350135+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:53:32.450578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352424+0000","last_change":"2024-02-07T13:34:15.439337+0000","last_active":"2024-02-07T13:34:38.352424+0000","last_peered":"2024-02-07T13:34:38.352424+0000","last_clean":"2024-02-07T13:34:38.352424+0000","last_became_active":"2024-02-07T13:34:15.439001+0000","last_became_peered":"2024-02-07T13:34:15.439001+0000","last_unstale":"2024-02-07T13:34:38.352424+0000","last_undegraded":"2024-02-07T13:34:38.352424+0000","last_fullsized":"2024-02-07T13:34:38.352424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:13:18.996828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353395+0000","last_change":"2024-02-07T13:34:11.420536+0000","last_active":"2024-02-07T13:34:38.353395+0000","last_peered":"2024-02-07T13:34:38.353395+0000","last_clean":"2024-02-07T13:34:38.353395+0000","last_became_active":"2024-02-07T13:34:11.419608+0000","last_became_peered":"2024-02-07T13:34:11.419608+0000","last_unstale":"2024-02-07T13:34:38.353395+0000","last_undegraded":"2024-02-07T13:34:38.353395+0000","last_fullsized":"2024-02-07T13:34:38.353395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:01:57.720890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353413+0000","last_change":"2024-02-07T13:34:13.429132+0000","last_active":"2024-02-07T13:34:38.353413+0000","last_peered":"2024-02-07T13:34:38.353413+0000","last_clean":"2024-02-07T13:34:38.353413+0000","last_became_active":"2024-02-07T13:34:13.428787+0000","last_became_peered":"2024-02-07T13:34:13.428787+0000","last_unstale":"2024-02-07T13:34:38.353413+0000","last_undegraded":"2024-02-07T13:34:38.353413+0000","last_fullsized":"2024-02-07T13:34:38.353413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:59:50.143822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355262+0000","last_change":"2024-02-07T13:34:17.448969+0000","last_active":"2024-02-07T13:34:38.355262+0000","last_peered":"2024-02-07T13:34:38.355262+0000","last_clean":"2024-02-07T13:34:38.355262+0000","last_became_active":"2024-02-07T13:34:17.448042+0000","last_became_peered":"2024-02-07T13:34:17.448042+0000","last_unstale":"2024-02-07T13:34:38.355262+0000","last_undegraded":"2024-02-07T13:34:38.355262+0000","last_fullsized":"2024-02-07T13:34:38.355262+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:57:59.770390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352686+0000","last_change":"2024-02-07T13:34:15.439656+0000","last_active":"2024-02-07T13:34:38.352686+0000","last_peered":"2024-02-07T13:34:38.352686+0000","last_clean":"2024-02-07T13:34:38.352686+0000","last_became_active":"2024-02-07T13:34:15.439243+0000","last_became_peered":"2024-02-07T13:34:15.439243+0000","last_unstale":"2024-02-07T13:34:38.352686+0000","last_undegraded":"2024-02-07T13:34:38.352686+0000","last_fullsized":"2024-02-07T13:34:38.352686+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:53:07.879042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357094+0000","last_change":"2024-02-07T13:34:11.420151+0000","last_active":"2024-02-07T13:34:38.357094+0000","last_peered":"2024-02-07T13:34:38.357094+0000","last_clean":"2024-02-07T13:34:38.357094+0000","last_became_active":"2024-02-07T13:34:11.419334+0000","last_became_peered":"2024-02-07T13:34:11.419334+0000","last_unstale":"2024-02-07T13:34:38.357094+0000","last_undegraded":"2024-02-07T13:34:38.357094+0000","last_fullsized":"2024-02-07T13:34:38.357094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:52:37.194909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355328+0000","last_change":"2024-02-07T13:34:13.430098+0000","last_active":"2024-02-07T13:34:38.355328+0000","last_peered":"2024-02-07T13:34:38.355328+0000","last_clean":"2024-02-07T13:34:38.355328+0000","last_became_active":"2024-02-07T13:34:13.429726+0000","last_became_peered":"2024-02-07T13:34:13.429726+0000","last_unstale":"2024-02-07T13:34:38.355328+0000","last_undegraded":"2024-02-07T13:34:38.355328+0000","last_fullsized":"2024-02-07T13:34:38.355328+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:07:05.787118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353005+0000","last_change":"2024-02-07T13:34:17.445312+0000","last_active":"2024-02-07T13:34:38.353005+0000","last_peered":"2024-02-07T13:34:38.353005+0000","last_clean":"2024-02-07T13:34:38.353005+0000","last_became_active":"2024-02-07T13:34:17.444746+0000","last_became_peered":"2024-02-07T13:34:17.444746+0000","last_unstale":"2024-02-07T13:34:38.353005+0000","last_undegraded":"2024-02-07T13:34:38.353005+0000","last_fullsized":"2024-02-07T13:34:38.353005+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:41:51.995066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352344+0000","last_change":"2024-02-07T13:34:15.437222+0000","last_active":"2024-02-07T13:34:38.352344+0000","last_peered":"2024-02-07T13:34:38.352344+0000","last_clean":"2024-02-07T13:34:38.352344+0000","last_became_active":"2024-02-07T13:34:15.436816+0000","last_became_peered":"2024-02-07T13:34:15.436816+0000","last_unstale":"2024-02-07T13:34:38.352344+0000","last_undegraded":"2024-02-07T13:34:38.352344+0000","last_fullsized":"2024-02-07T13:34:38.352344+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:41:20.282903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355586+0000","last_change":"2024-02-07T13:34:11.423934+0000","last_active":"2024-02-07T13:34:38.355586+0000","last_peered":"2024-02-07T13:34:38.355586+0000","last_clean":"2024-02-07T13:34:38.355586+0000","last_became_active":"2024-02-07T13:34:11.423615+0000","last_became_peered":"2024-02-07T13:34:11.423615+0000","last_unstale":"2024-02-07T13:34:38.355586+0000","last_undegraded":"2024-02-07T13:34:38.355586+0000","last_fullsized":"2024-02-07T13:34:38.355586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:53:53.398276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356984+0000","last_change":"2024-02-07T13:34:13.435374+0000","last_active":"2024-02-07T13:34:38.356984+0000","last_peered":"2024-02-07T13:34:38.356984+0000","last_clean":"2024-02-07T13:34:38.356984+0000","last_became_active":"2024-02-07T13:34:13.435086+0000","last_became_peered":"2024-02-07T13:34:13.435086+0000","last_unstale":"2024-02-07T13:34:38.356984+0000","last_undegraded":"2024-02-07T13:34:38.356984+0000","last_fullsized":"2024-02-07T13:34:38.356984+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:06:39.164361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354876+0000","last_change":"2024-02-07T13:34:17.447806+0000","last_active":"2024-02-07T13:34:38.354876+0000","last_peered":"2024-02-07T13:34:38.354876+0000","last_clean":"2024-02-07T13:34:38.354876+0000","last_became_active":"2024-02-07T13:34:17.446951+0000","last_became_peered":"2024-02-07T13:34:17.446951+0000","last_unstale":"2024-02-07T13:34:38.354876+0000","last_undegraded":"2024-02-07T13:34:38.354876+0000","last_fullsized":"2024-02-07T13:34:38.354876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:02:08.290401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356184+0000","last_change":"2024-02-07T13:34:15.433608+0000","last_active":"2024-02-07T13:34:38.356184+0000","last_peered":"2024-02-07T13:34:38.356184+0000","last_clean":"2024-02-07T13:34:38.356184+0000","last_became_active":"2024-02-07T13:34:15.432492+0000","last_became_peered":"2024-02-07T13:34:15.432492+0000","last_unstale":"2024-02-07T13:34:38.356184+0000","last_undegraded":"2024-02-07T13:34:38.356184+0000","last_fullsized":"2024-02-07T13:34:38.356184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:47:33.155614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356646+0000","last_change":"2024-02-07T13:34:11.420795+0000","last_active":"2024-02-07T13:34:38.356646+0000","last_peered":"2024-02-07T13:34:38.356646+0000","last_clean":"2024-02-07T13:34:38.356646+0000","last_became_active":"2024-02-07T13:34:11.420051+0000","last_became_peered":"2024-02-07T13:34:11.420051+0000","last_unstale":"2024-02-07T13:34:38.356646+0000","last_undegraded":"2024-02-07T13:34:38.356646+0000","last_fullsized":"2024-02-07T13:34:38.356646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:08:42.458805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355995+0000","last_change":"2024-02-07T13:34:13.435150+0000","last_active":"2024-02-07T13:34:38.355995+0000","last_peered":"2024-02-07T13:34:38.355995+0000","last_clean":"2024-02-07T13:34:38.355995+0000","last_became_active":"2024-02-07T13:34:13.434624+0000","last_became_peered":"2024-02-07T13:34:13.434624+0000","last_unstale":"2024-02-07T13:34:38.355995+0000","last_undegraded":"2024-02-07T13:34:38.355995+0000","last_fullsized":"2024-02-07T13:34:38.355995+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:22:30.337210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352252+0000","last_change":"2024-02-07T13:34:17.448906+0000","last_active":"2024-02-07T13:34:38.352252+0000","last_peered":"2024-02-07T13:34:38.352252+0000","last_clean":"2024-02-07T13:34:38.352252+0000","last_became_active":"2024-02-07T13:34:17.448536+0000","last_became_peered":"2024-02-07T13:34:17.448536+0000","last_unstale":"2024-02-07T13:34:38.352252+0000","last_undegraded":"2024-02-07T13:34:38.352252+0000","last_fullsized":"2024-02-07T13:34:38.352252+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:17:02.360141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355841+0000","last_change":"2024-02-07T13:34:15.432160+0000","last_active":"2024-02-07T13:34:38.355841+0000","last_peered":"2024-02-07T13:34:38.355841+0000","last_clean":"2024-02-07T13:34:38.355841+0000","last_became_active":"2024-02-07T13:34:15.431887+0000","last_became_peered":"2024-02-07T13:34:15.431887+0000","last_unstale":"2024-02-07T13:34:38.355841+0000","last_undegraded":"2024-02-07T13:34:38.355841+0000","last_fullsized":"2024-02-07T13:34:38.355841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:03:30.073273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352808+0000","last_change":"2024-02-07T13:34:11.419829+0000","last_active":"2024-02-07T13:34:38.352808+0000","last_peered":"2024-02-07T13:34:38.352808+0000","last_clean":"2024-02-07T13:34:38.352808+0000","last_became_active":"2024-02-07T13:34:11.419443+0000","last_became_peered":"2024-02-07T13:34:11.419443+0000","last_unstale":"2024-02-07T13:34:38.352808+0000","last_undegraded":"2024-02-07T13:34:38.352808+0000","last_fullsized":"2024-02-07T13:34:38.352808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:59:25.520579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356132+0000","last_change":"2024-02-07T13:34:13.432437+0000","last_active":"2024-02-07T13:34:38.356132+0000","last_peered":"2024-02-07T13:34:38.356132+0000","last_clean":"2024-02-07T13:34:38.356132+0000","last_became_active":"2024-02-07T13:34:13.431768+0000","last_became_peered":"2024-02-07T13:34:13.431768+0000","last_unstale":"2024-02-07T13:34:38.356132+0000","last_undegraded":"2024-02-07T13:34:38.356132+0000","last_fullsized":"2024-02-07T13:34:38.356132+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:03:28.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.351017+0000","last_change":"2024-02-07T13:34:17.449510+0000","last_active":"2024-02-07T13:34:38.351017+0000","last_peered":"2024-02-07T13:34:38.351017+0000","last_clean":"2024-02-07T13:34:38.351017+0000","last_became_active":"2024-02-07T13:34:17.449156+0000","last_became_peered":"2024-02-07T13:34:17.449156+0000","last_unstale":"2024-02-07T13:34:38.351017+0000","last_undegraded":"2024-02-07T13:34:38.351017+0000","last_fullsized":"2024-02-07T13:34:38.351017+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:50:37.678973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468297+0000","last_change":"2024-02-07T13:34:15.434396+0000","last_active":"2024-02-07T13:34:41.468297+0000","last_peered":"2024-02-07T13:34:41.468297+0000","last_clean":"2024-02-07T13:34:41.468297+0000","last_became_active":"2024-02-07T13:34:15.433652+0000","last_became_peered":"2024-02-07T13:34:15.433652+0000","last_unstale":"2024-02-07T13:34:41.468297+0000","last_undegraded":"2024-02-07T13:34:41.468297+0000","last_fullsized":"2024-02-07T13:34:41.468297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:58:59.586215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350574+0000","last_change":"2024-02-07T13:34:11.420820+0000","last_active":"2024-02-07T13:34:38.350574+0000","last_peered":"2024-02-07T13:34:38.350574+0000","last_clean":"2024-02-07T13:34:38.350574+0000","last_became_active":"2024-02-07T13:34:11.420009+0000","last_became_peered":"2024-02-07T13:34:11.420009+0000","last_unstale":"2024-02-07T13:34:38.350574+0000","last_undegraded":"2024-02-07T13:34:38.350574+0000","last_fullsized":"2024-02-07T13:34:38.350574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:20:49.141845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":920,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.349981+0000","last_change":"2024-02-07T13:34:13.430135+0000","last_active":"2024-02-07T13:34:38.349981+0000","last_peered":"2024-02-07T13:34:38.349981+0000","last_clean":"2024-02-07T13:34:38.349981+0000","last_became_active":"2024-02-07T13:34:13.428963+0000","last_became_peered":"2024-02-07T13:34:13.428963+0000","last_unstale":"2024-02-07T13:34:38.349981+0000","last_undegraded":"2024-02-07T13:34:38.349981+0000","last_fullsized":"2024-02-07T13:34:38.349981+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:37:26.784644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353781+0000","last_change":"2024-02-07T13:34:17.448344+0000","last_active":"2024-02-07T13:34:38.353781+0000","last_peered":"2024-02-07T13:34:38.353781+0000","last_clean":"2024-02-07T13:34:38.353781+0000","last_became_active":"2024-02-07T13:34:17.447797+0000","last_became_peered":"2024-02-07T13:34:17.447797+0000","last_unstale":"2024-02-07T13:34:38.353781+0000","last_undegraded":"2024-02-07T13:34:38.353781+0000","last_fullsized":"2024-02-07T13:34:38.353781+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:09:14.154285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352598+0000","last_change":"2024-02-07T13:34:15.434361+0000","last_active":"2024-02-07T13:34:38.352598+0000","last_peered":"2024-02-07T13:34:38.352598+0000","last_clean":"2024-02-07T13:34:38.352598+0000","last_became_active":"2024-02-07T13:34:15.433807+0000","last_became_peered":"2024-02-07T13:34:15.433807+0000","last_unstale":"2024-02-07T13:34:38.352598+0000","last_undegraded":"2024-02-07T13:34:38.352598+0000","last_fullsized":"2024-02-07T13:34:38.352598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:03:20.395174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354966+0000","last_change":"2024-02-07T13:34:11.420629+0000","last_active":"2024-02-07T13:34:38.354966+0000","last_peered":"2024-02-07T13:34:38.354966+0000","last_clean":"2024-02-07T13:34:38.354966+0000","last_became_active":"2024-02-07T13:34:11.420329+0000","last_became_peered":"2024-02-07T13:34:11.420329+0000","last_unstale":"2024-02-07T13:34:38.354966+0000","last_undegraded":"2024-02-07T13:34:38.354966+0000","last_fullsized":"2024-02-07T13:34:38.354966+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:29:19.756265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.349998+0000","last_change":"2024-02-07T13:34:13.430211+0000","last_active":"2024-02-07T13:34:38.349998+0000","last_peered":"2024-02-07T13:34:38.349998+0000","last_clean":"2024-02-07T13:34:38.349998+0000","last_became_active":"2024-02-07T13:34:13.428474+0000","last_became_peered":"2024-02-07T13:34:13.428474+0000","last_unstale":"2024-02-07T13:34:38.349998+0000","last_undegraded":"2024-02-07T13:34:38.349998+0000","last_fullsized":"2024-02-07T13:34:38.349998+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:40:27.223364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352851+0000","last_change":"2024-02-07T13:34:17.446797+0000","last_active":"2024-02-07T13:34:38.352851+0000","last_peered":"2024-02-07T13:34:38.352851+0000","last_clean":"2024-02-07T13:34:38.352851+0000","last_became_active":"2024-02-07T13:34:17.446445+0000","last_became_peered":"2024-02-07T13:34:17.446445+0000","last_unstale":"2024-02-07T13:34:38.352851+0000","last_undegraded":"2024-02-07T13:34:38.352851+0000","last_fullsized":"2024-02-07T13:34:38.352851+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:36:24.011394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.349893+0000","last_change":"2024-02-07T13:34:15.434239+0000","last_active":"2024-02-07T13:34:38.349893+0000","last_peered":"2024-02-07T13:34:38.349893+0000","last_clean":"2024-02-07T13:34:38.349893+0000","last_became_active":"2024-02-07T13:34:15.433878+0000","last_became_peered":"2024-02-07T13:34:15.433878+0000","last_unstale":"2024-02-07T13:34:38.349893+0000","last_undegraded":"2024-02-07T13:34:38.349893+0000","last_fullsized":"2024-02-07T13:34:38.349893+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:25:57.461991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354964+0000","last_change":"2024-02-07T13:34:11.424045+0000","last_active":"2024-02-07T13:34:38.354964+0000","last_peered":"2024-02-07T13:34:38.354964+0000","last_clean":"2024-02-07T13:34:38.354964+0000","last_became_active":"2024-02-07T13:34:11.423733+0000","last_became_peered":"2024-02-07T13:34:11.423733+0000","last_unstale":"2024-02-07T13:34:38.354964+0000","last_undegraded":"2024-02-07T13:34:38.354964+0000","last_fullsized":"2024-02-07T13:34:38.354964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:53:57.469051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353767+0000","last_change":"2024-02-07T13:34:13.429172+0000","last_active":"2024-02-07T13:34:38.353767+0000","last_peered":"2024-02-07T13:34:38.353767+0000","last_clean":"2024-02-07T13:34:38.353767+0000","last_became_active":"2024-02-07T13:34:13.428849+0000","last_became_peered":"2024-02-07T13:34:13.428849+0000","last_unstale":"2024-02-07T13:34:38.353767+0000","last_undegraded":"2024-02-07T13:34:38.353767+0000","last_fullsized":"2024-02-07T13:34:38.353767+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:05:08.010111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352519+0000","last_change":"2024-02-07T13:34:17.447233+0000","last_active":"2024-02-07T13:34:38.352519+0000","last_peered":"2024-02-07T13:34:38.352519+0000","last_clean":"2024-02-07T13:34:38.352519+0000","last_became_active":"2024-02-07T13:34:17.446406+0000","last_became_peered":"2024-02-07T13:34:17.446406+0000","last_unstale":"2024-02-07T13:34:38.352519+0000","last_undegraded":"2024-02-07T13:34:38.352519+0000","last_fullsized":"2024-02-07T13:34:38.352519+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:00:00.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350441+0000","last_change":"2024-02-07T13:34:15.436466+0000","last_active":"2024-02-07T13:34:38.350441+0000","last_peered":"2024-02-07T13:34:38.350441+0000","last_clean":"2024-02-07T13:34:38.350441+0000","last_became_active":"2024-02-07T13:34:15.436081+0000","last_became_peered":"2024-02-07T13:34:15.436081+0000","last_unstale":"2024-02-07T13:34:38.350441+0000","last_undegraded":"2024-02-07T13:34:38.350441+0000","last_fullsized":"2024-02-07T13:34:38.350441+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:57:44.431175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353601+0000","last_change":"2024-02-07T13:34:11.424461+0000","last_active":"2024-02-07T13:34:38.353601+0000","last_peered":"2024-02-07T13:34:38.353601+0000","last_clean":"2024-02-07T13:34:38.353601+0000","last_became_active":"2024-02-07T13:34:11.424092+0000","last_became_peered":"2024-02-07T13:34:11.424092+0000","last_unstale":"2024-02-07T13:34:38.353601+0000","last_undegraded":"2024-02-07T13:34:38.353601+0000","last_fullsized":"2024-02-07T13:34:38.353601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:42:59.389803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354771+0000","last_change":"2024-02-07T13:34:13.430371+0000","last_active":"2024-02-07T13:34:38.354771+0000","last_peered":"2024-02-07T13:34:38.354771+0000","last_clean":"2024-02-07T13:34:38.354771+0000","last_became_active":"2024-02-07T13:34:13.430062+0000","last_became_peered":"2024-02-07T13:34:13.430062+0000","last_unstale":"2024-02-07T13:34:38.354771+0000","last_undegraded":"2024-02-07T13:34:38.354771+0000","last_fullsized":"2024-02-07T13:34:38.354771+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:22:43.486630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355366+0000","last_change":"2024-02-07T13:34:17.448727+0000","last_active":"2024-02-07T13:34:38.355366+0000","last_peered":"2024-02-07T13:34:38.355366+0000","last_clean":"2024-02-07T13:34:38.355366+0000","last_became_active":"2024-02-07T13:34:17.448423+0000","last_became_peered":"2024-02-07T13:34:17.448423+0000","last_unstale":"2024-02-07T13:34:38.355366+0000","last_undegraded":"2024-02-07T13:34:38.355366+0000","last_fullsized":"2024-02-07T13:34:38.355366+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:01:44.308094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353176+0000","last_change":"2024-02-07T13:34:15.437267+0000","last_active":"2024-02-07T13:34:38.353176+0000","last_peered":"2024-02-07T13:34:38.353176+0000","last_clean":"2024-02-07T13:34:38.353176+0000","last_became_active":"2024-02-07T13:34:15.436792+0000","last_became_peered":"2024-02-07T13:34:15.436792+0000","last_unstale":"2024-02-07T13:34:38.353176+0000","last_undegraded":"2024-02-07T13:34:38.353176+0000","last_fullsized":"2024-02-07T13:34:38.353176+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:39:52.476866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356947+0000","last_change":"2024-02-07T13:34:11.420689+0000","last_active":"2024-02-07T13:34:38.356947+0000","last_peered":"2024-02-07T13:34:38.356947+0000","last_clean":"2024-02-07T13:34:38.356947+0000","last_became_active":"2024-02-07T13:34:11.419927+0000","last_became_peered":"2024-02-07T13:34:11.419927+0000","last_unstale":"2024-02-07T13:34:38.356947+0000","last_undegraded":"2024-02-07T13:34:38.356947+0000","last_fullsized":"2024-02-07T13:34:38.356947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:09:19.994316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355167+0000","last_change":"2024-02-07T13:34:13.427809+0000","last_active":"2024-02-07T13:34:38.355167+0000","last_peered":"2024-02-07T13:34:38.355167+0000","last_clean":"2024-02-07T13:34:38.355167+0000","last_became_active":"2024-02-07T13:34:13.427487+0000","last_became_peered":"2024-02-07T13:34:13.427487+0000","last_unstale":"2024-02-07T13:34:38.355167+0000","last_undegraded":"2024-02-07T13:34:38.355167+0000","last_fullsized":"2024-02-07T13:34:38.355167+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:55:06.630474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355372+0000","last_change":"2024-02-07T13:34:17.448846+0000","last_active":"2024-02-07T13:34:38.355372+0000","last_peered":"2024-02-07T13:34:38.355372+0000","last_clean":"2024-02-07T13:34:38.355372+0000","last_became_active":"2024-02-07T13:34:17.448603+0000","last_became_peered":"2024-02-07T13:34:17.448603+0000","last_unstale":"2024-02-07T13:34:38.355372+0000","last_undegraded":"2024-02-07T13:34:38.355372+0000","last_fullsized":"2024-02-07T13:34:38.355372+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:45:58.102122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350056+0000","last_change":"2024-02-07T13:34:15.434590+0000","last_active":"2024-02-07T13:34:38.350056+0000","last_peered":"2024-02-07T13:34:38.350056+0000","last_clean":"2024-02-07T13:34:38.350056+0000","last_became_active":"2024-02-07T13:34:15.434268+0000","last_became_peered":"2024-02-07T13:34:15.434268+0000","last_unstale":"2024-02-07T13:34:38.350056+0000","last_undegraded":"2024-02-07T13:34:38.350056+0000","last_fullsized":"2024-02-07T13:34:38.350056+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:24:00.707025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355077+0000","last_change":"2024-02-07T13:34:11.424011+0000","last_active":"2024-02-07T13:34:38.355077+0000","last_peered":"2024-02-07T13:34:38.355077+0000","last_clean":"2024-02-07T13:34:38.355077+0000","last_became_active":"2024-02-07T13:34:11.423566+0000","last_became_peered":"2024-02-07T13:34:11.423566+0000","last_unstale":"2024-02-07T13:34:38.355077+0000","last_undegraded":"2024-02-07T13:34:38.355077+0000","last_fullsized":"2024-02-07T13:34:38.355077+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:32:37.801622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356743+0000","last_change":"2024-02-07T13:34:13.435131+0000","last_active":"2024-02-07T13:34:38.356743+0000","last_peered":"2024-02-07T13:34:38.356743+0000","last_clean":"2024-02-07T13:34:38.356743+0000","last_became_active":"2024-02-07T13:34:13.434670+0000","last_became_peered":"2024-02-07T13:34:13.434670+0000","last_unstale":"2024-02-07T13:34:38.356743+0000","last_undegraded":"2024-02-07T13:34:38.356743+0000","last_fullsized":"2024-02-07T13:34:38.356743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:25:26.379511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352880+0000","last_change":"2024-02-07T13:34:17.446281+0000","last_active":"2024-02-07T13:34:38.352880+0000","last_peered":"2024-02-07T13:34:38.352880+0000","last_clean":"2024-02-07T13:34:38.352880+0000","last_became_active":"2024-02-07T13:34:17.445870+0000","last_became_peered":"2024-02-07T13:34:17.445870+0000","last_unstale":"2024-02-07T13:34:38.352880+0000","last_undegraded":"2024-02-07T13:34:38.352880+0000","last_fullsized":"2024-02-07T13:34:38.352880+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:35:26.733830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468345+0000","last_change":"2024-02-07T13:34:15.432427+0000","last_active":"2024-02-07T13:34:41.468345+0000","last_peered":"2024-02-07T13:34:41.468345+0000","last_clean":"2024-02-07T13:34:41.468345+0000","last_became_active":"2024-02-07T13:34:15.432134+0000","last_became_peered":"2024-02-07T13:34:15.432134+0000","last_unstale":"2024-02-07T13:34:41.468345+0000","last_undegraded":"2024-02-07T13:34:41.468345+0000","last_fullsized":"2024-02-07T13:34:41.468345+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:15:50.805974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355902+0000","last_change":"2024-02-07T13:34:11.420900+0000","last_active":"2024-02-07T13:34:38.355902+0000","last_peered":"2024-02-07T13:34:38.355902+0000","last_clean":"2024-02-07T13:34:38.355902+0000","last_became_active":"2024-02-07T13:34:11.420383+0000","last_became_peered":"2024-02-07T13:34:11.420383+0000","last_unstale":"2024-02-07T13:34:38.355902+0000","last_undegraded":"2024-02-07T13:34:38.355902+0000","last_fullsized":"2024-02-07T13:34:38.355902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:58:47.580443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350442+0000","last_change":"2024-02-07T13:34:13.427434+0000","last_active":"2024-02-07T13:34:38.350442+0000","last_peered":"2024-02-07T13:34:38.350442+0000","last_clean":"2024-02-07T13:34:38.350442+0000","last_became_active":"2024-02-07T13:34:13.427074+0000","last_became_peered":"2024-02-07T13:34:13.427074+0000","last_unstale":"2024-02-07T13:34:38.350442+0000","last_undegraded":"2024-02-07T13:34:38.350442+0000","last_fullsized":"2024-02-07T13:34:38.350442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:04:47.455644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352933+0000","last_change":"2024-02-07T13:34:17.447043+0000","last_active":"2024-02-07T13:34:38.352933+0000","last_peered":"2024-02-07T13:34:38.352933+0000","last_clean":"2024-02-07T13:34:38.352933+0000","last_became_active":"2024-02-07T13:34:17.446227+0000","last_became_peered":"2024-02-07T13:34:17.446227+0000","last_unstale":"2024-02-07T13:34:38.352933+0000","last_undegraded":"2024-02-07T13:34:38.352933+0000","last_fullsized":"2024-02-07T13:34:38.352933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:29:56.967993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468339+0000","last_change":"2024-02-07T13:34:15.433876+0000","last_active":"2024-02-07T13:34:41.468339+0000","last_peered":"2024-02-07T13:34:41.468339+0000","last_clean":"2024-02-07T13:34:41.468339+0000","last_became_active":"2024-02-07T13:34:15.432776+0000","last_became_peered":"2024-02-07T13:34:15.432776+0000","last_unstale":"2024-02-07T13:34:41.468339+0000","last_undegraded":"2024-02-07T13:34:41.468339+0000","last_fullsized":"2024-02-07T13:34:41.468339+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:39:46.118469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355610+0000","last_change":"2024-02-07T13:34:11.419418+0000","last_active":"2024-02-07T13:34:38.355610+0000","last_peered":"2024-02-07T13:34:38.355610+0000","last_clean":"2024-02-07T13:34:38.355610+0000","last_became_active":"2024-02-07T13:34:11.419046+0000","last_became_peered":"2024-02-07T13:34:11.419046+0000","last_unstale":"2024-02-07T13:34:38.355610+0000","last_undegraded":"2024-02-07T13:34:38.355610+0000","last_fullsized":"2024-02-07T13:34:38.355610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:28:35.600401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353389+0000","last_change":"2024-02-07T13:34:13.428022+0000","last_active":"2024-02-07T13:34:38.353389+0000","last_peered":"2024-02-07T13:34:38.353389+0000","last_clean":"2024-02-07T13:34:38.353389+0000","last_became_active":"2024-02-07T13:34:13.427623+0000","last_became_peered":"2024-02-07T13:34:13.427623+0000","last_unstale":"2024-02-07T13:34:38.353389+0000","last_undegraded":"2024-02-07T13:34:38.353389+0000","last_fullsized":"2024-02-07T13:34:38.353389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:43:06.553569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352934+0000","last_change":"2024-02-07T13:34:17.447973+0000","last_active":"2024-02-07T13:34:38.352934+0000","last_peered":"2024-02-07T13:34:38.352934+0000","last_clean":"2024-02-07T13:34:38.352934+0000","last_became_active":"2024-02-07T13:34:17.446117+0000","last_became_peered":"2024-02-07T13:34:17.446117+0000","last_unstale":"2024-02-07T13:34:38.352934+0000","last_undegraded":"2024-02-07T13:34:38.352934+0000","last_fullsized":"2024-02-07T13:34:38.352934+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:15:28.589183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353243+0000","last_change":"2024-02-07T13:34:15.435544+0000","last_active":"2024-02-07T13:34:38.353243+0000","last_peered":"2024-02-07T13:34:38.353243+0000","last_clean":"2024-02-07T13:34:38.353243+0000","last_became_active":"2024-02-07T13:34:15.434899+0000","last_became_peered":"2024-02-07T13:34:15.434899+0000","last_unstale":"2024-02-07T13:34:38.353243+0000","last_undegraded":"2024-02-07T13:34:38.353243+0000","last_fullsized":"2024-02-07T13:34:38.353243+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:33:55.440321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352417+0000","last_change":"2024-02-07T13:34:11.419779+0000","last_active":"2024-02-07T13:34:38.352417+0000","last_peered":"2024-02-07T13:34:38.352417+0000","last_clean":"2024-02-07T13:34:38.352417+0000","last_became_active":"2024-02-07T13:34:11.419445+0000","last_became_peered":"2024-02-07T13:34:11.419445+0000","last_unstale":"2024-02-07T13:34:38.352417+0000","last_undegraded":"2024-02-07T13:34:38.352417+0000","last_fullsized":"2024-02-07T13:34:38.352417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:32:12.627238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352911+0000","last_change":"2024-02-07T13:34:13.429901+0000","last_active":"2024-02-07T13:34:38.352911+0000","last_peered":"2024-02-07T13:34:38.352911+0000","last_clean":"2024-02-07T13:34:38.352911+0000","last_became_active":"2024-02-07T13:34:13.428705+0000","last_became_peered":"2024-02-07T13:34:13.428705+0000","last_unstale":"2024-02-07T13:34:38.352911+0000","last_undegraded":"2024-02-07T13:34:38.352911+0000","last_fullsized":"2024-02-07T13:34:38.352911+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:17:00.563014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352981+0000","last_change":"2024-02-07T13:34:17.446627+0000","last_active":"2024-02-07T13:34:38.352981+0000","last_peered":"2024-02-07T13:34:38.352981+0000","last_clean":"2024-02-07T13:34:38.352981+0000","last_became_active":"2024-02-07T13:34:17.446175+0000","last_became_peered":"2024-02-07T13:34:17.446175+0000","last_unstale":"2024-02-07T13:34:38.352981+0000","last_undegraded":"2024-02-07T13:34:38.352981+0000","last_fullsized":"2024-02-07T13:34:38.352981+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:54:10.152159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352678+0000","last_change":"2024-02-07T13:34:15.435390+0000","last_active":"2024-02-07T13:34:38.352678+0000","last_peered":"2024-02-07T13:34:38.352678+0000","last_clean":"2024-02-07T13:34:38.352678+0000","last_became_active":"2024-02-07T13:34:15.434805+0000","last_became_peered":"2024-02-07T13:34:15.434805+0000","last_unstale":"2024-02-07T13:34:38.352678+0000","last_undegraded":"2024-02-07T13:34:38.352678+0000","last_fullsized":"2024-02-07T13:34:38.352678+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:56:30.098575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355137+0000","last_change":"2024-02-07T13:34:11.425143+0000","last_active":"2024-02-07T13:34:38.355137+0000","last_peered":"2024-02-07T13:34:38.355137+0000","last_clean":"2024-02-07T13:34:38.355137+0000","last_became_active":"2024-02-07T13:34:11.424648+0000","last_became_peered":"2024-02-07T13:34:11.424648+0000","last_unstale":"2024-02-07T13:34:38.355137+0000","last_undegraded":"2024-02-07T13:34:38.355137+0000","last_fullsized":"2024-02-07T13:34:38.355137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:54:57.242656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355112+0000","last_change":"2024-02-07T13:34:13.432344+0000","last_active":"2024-02-07T13:34:38.355112+0000","last_peered":"2024-02-07T13:34:38.355112+0000","last_clean":"2024-02-07T13:34:38.355112+0000","last_became_active":"2024-02-07T13:34:13.431703+0000","last_became_peered":"2024-02-07T13:34:13.431703+0000","last_unstale":"2024-02-07T13:34:38.355112+0000","last_undegraded":"2024-02-07T13:34:38.355112+0000","last_fullsized":"2024-02-07T13:34:38.355112+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:57:56.365302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355722+0000","last_change":"2024-02-07T13:34:17.448487+0000","last_active":"2024-02-07T13:34:38.355722+0000","last_peered":"2024-02-07T13:34:38.355722+0000","last_clean":"2024-02-07T13:34:38.355722+0000","last_became_active":"2024-02-07T13:34:17.448238+0000","last_became_peered":"2024-02-07T13:34:17.448238+0000","last_unstale":"2024-02-07T13:34:38.355722+0000","last_undegraded":"2024-02-07T13:34:38.355722+0000","last_fullsized":"2024-02-07T13:34:38.355722+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:29:30.060230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357668+0000","last_change":"2024-02-07T13:34:11.419648+0000","last_active":"2024-02-07T13:34:38.357668+0000","last_peered":"2024-02-07T13:34:38.357668+0000","last_clean":"2024-02-07T13:34:38.357668+0000","last_became_active":"2024-02-07T13:34:11.419041+0000","last_became_peered":"2024-02-07T13:34:11.419041+0000","last_unstale":"2024-02-07T13:34:38.357668+0000","last_undegraded":"2024-02-07T13:34:38.357668+0000","last_fullsized":"2024-02-07T13:34:38.357668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:36:57.130050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357413+0000","last_change":"2024-02-07T13:34:13.435296+0000","last_active":"2024-02-07T13:34:38.357413+0000","last_peered":"2024-02-07T13:34:38.357413+0000","last_clean":"2024-02-07T13:34:38.357413+0000","last_became_active":"2024-02-07T13:34:13.435017+0000","last_became_peered":"2024-02-07T13:34:13.435017+0000","last_unstale":"2024-02-07T13:34:38.357413+0000","last_undegraded":"2024-02-07T13:34:38.357413+0000","last_fullsized":"2024-02-07T13:34:38.357413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:34:00.419145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468370+0000","last_change":"2024-02-07T13:34:15.432735+0000","last_active":"2024-02-07T13:34:41.468370+0000","last_peered":"2024-02-07T13:34:41.468370+0000","last_clean":"2024-02-07T13:34:41.468370+0000","last_became_active":"2024-02-07T13:34:15.432451+0000","last_became_peered":"2024-02-07T13:34:15.432451+0000","last_unstale":"2024-02-07T13:34:41.468370+0000","last_undegraded":"2024-02-07T13:34:41.468370+0000","last_fullsized":"2024-02-07T13:34:41.468370+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:26:19.853005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350352+0000","last_change":"2024-02-07T13:34:17.444006+0000","last_active":"2024-02-07T13:34:38.350352+0000","last_peered":"2024-02-07T13:34:38.350352+0000","last_clean":"2024-02-07T13:34:38.350352+0000","last_became_active":"2024-02-07T13:34:17.443701+0000","last_became_peered":"2024-02-07T13:34:17.443701+0000","last_unstale":"2024-02-07T13:34:38.350352+0000","last_undegraded":"2024-02-07T13:34:38.350352+0000","last_fullsized":"2024-02-07T13:34:38.350352+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:05:09.757974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353127+0000","last_change":"2024-02-07T13:34:11.420704+0000","last_active":"2024-02-07T13:34:38.353127+0000","last_peered":"2024-02-07T13:34:38.353127+0000","last_clean":"2024-02-07T13:34:38.353127+0000","last_became_active":"2024-02-07T13:34:11.420266+0000","last_became_peered":"2024-02-07T13:34:11.420266+0000","last_unstale":"2024-02-07T13:34:38.353127+0000","last_undegraded":"2024-02-07T13:34:38.353127+0000","last_fullsized":"2024-02-07T13:34:38.353127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:04:01.787605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468133+0000","last_change":"2024-02-07T13:34:13.429780+0000","last_active":"2024-02-07T13:34:41.468133+0000","last_peered":"2024-02-07T13:34:41.468133+0000","last_clean":"2024-02-07T13:34:41.468133+0000","last_became_active":"2024-02-07T13:34:13.428647+0000","last_became_peered":"2024-02-07T13:34:13.428647+0000","last_unstale":"2024-02-07T13:34:41.468133+0000","last_undegraded":"2024-02-07T13:34:41.468133+0000","last_fullsized":"2024-02-07T13:34:41.468133+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:04:53.190980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355460+0000","last_change":"2024-02-07T13:34:15.430738+0000","last_active":"2024-02-07T13:34:38.355460+0000","last_peered":"2024-02-07T13:34:38.355460+0000","last_clean":"2024-02-07T13:34:38.355460+0000","last_became_active":"2024-02-07T13:34:15.430476+0000","last_became_peered":"2024-02-07T13:34:15.430476+0000","last_unstale":"2024-02-07T13:34:38.355460+0000","last_undegraded":"2024-02-07T13:34:38.355460+0000","last_fullsized":"2024-02-07T13:34:38.355460+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:34:26.724082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350821+0000","last_change":"2024-02-07T13:34:17.446368+0000","last_active":"2024-02-07T13:34:38.350821+0000","last_peered":"2024-02-07T13:34:38.350821+0000","last_clean":"2024-02-07T13:34:38.350821+0000","last_became_active":"2024-02-07T13:34:17.446027+0000","last_became_peered":"2024-02-07T13:34:17.446027+0000","last_unstale":"2024-02-07T13:34:38.350821+0000","last_undegraded":"2024-02-07T13:34:38.350821+0000","last_fullsized":"2024-02-07T13:34:38.350821+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:02:56.944115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352902+0000","last_change":"2024-02-07T13:34:11.423264+0000","last_active":"2024-02-07T13:34:38.352902+0000","last_peered":"2024-02-07T13:34:38.352902+0000","last_clean":"2024-02-07T13:34:38.352902+0000","last_became_active":"2024-02-07T13:34:11.422917+0000","last_became_peered":"2024-02-07T13:34:11.422917+0000","last_unstale":"2024-02-07T13:34:38.352902+0000","last_undegraded":"2024-02-07T13:34:38.352902+0000","last_fullsized":"2024-02-07T13:34:38.352902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:35:25.720896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354734+0000","last_change":"2024-02-07T13:34:13.428004+0000","last_active":"2024-02-07T13:34:38.354734+0000","last_peered":"2024-02-07T13:34:38.354734+0000","last_clean":"2024-02-07T13:34:38.354734+0000","last_became_active":"2024-02-07T13:34:13.427660+0000","last_became_peered":"2024-02-07T13:34:13.427660+0000","last_unstale":"2024-02-07T13:34:38.354734+0000","last_undegraded":"2024-02-07T13:34:38.354734+0000","last_fullsized":"2024-02-07T13:34:38.354734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:14:38.543175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352973+0000","last_change":"2024-02-07T13:34:15.434731+0000","last_active":"2024-02-07T13:34:38.352973+0000","last_peered":"2024-02-07T13:34:38.352973+0000","last_clean":"2024-02-07T13:34:38.352973+0000","last_became_active":"2024-02-07T13:34:15.434387+0000","last_became_peered":"2024-02-07T13:34:15.434387+0000","last_unstale":"2024-02-07T13:34:38.352973+0000","last_undegraded":"2024-02-07T13:34:38.352973+0000","last_fullsized":"2024-02-07T13:34:38.352973+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:14:27.602695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353479+0000","last_change":"2024-02-07T13:34:17.449515+0000","last_active":"2024-02-07T13:34:38.353479+0000","last_peered":"2024-02-07T13:34:38.353479+0000","last_clean":"2024-02-07T13:34:38.353479+0000","last_became_active":"2024-02-07T13:34:17.449250+0000","last_became_peered":"2024-02-07T13:34:17.449250+0000","last_unstale":"2024-02-07T13:34:38.353479+0000","last_undegraded":"2024-02-07T13:34:38.353479+0000","last_fullsized":"2024-02-07T13:34:38.353479+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:49:58.354627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354807+0000","last_change":"2024-02-07T13:34:13.435045+0000","last_active":"2024-02-07T13:34:38.354807+0000","last_peered":"2024-02-07T13:34:38.354807+0000","last_clean":"2024-02-07T13:34:38.354807+0000","last_became_active":"2024-02-07T13:34:13.434538+0000","last_became_peered":"2024-02-07T13:34:13.434538+0000","last_unstale":"2024-02-07T13:34:38.354807+0000","last_undegraded":"2024-02-07T13:34:38.354807+0000","last_fullsized":"2024-02-07T13:34:38.354807+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:46:13.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350085+0000","last_change":"2024-02-07T13:34:11.422066+0000","last_active":"2024-02-07T13:34:38.350085+0000","last_peered":"2024-02-07T13:34:38.350085+0000","last_clean":"2024-02-07T13:34:38.350085+0000","last_became_active":"2024-02-07T13:34:11.420383+0000","last_became_peered":"2024-02-07T13:34:11.420383+0000","last_unstale":"2024-02-07T13:34:38.350085+0000","last_undegraded":"2024-02-07T13:34:38.350085+0000","last_fullsized":"2024-02-07T13:34:38.350085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:54:06.859637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354764+0000","last_change":"2024-02-07T13:34:15.436526+0000","last_active":"2024-02-07T13:34:38.354764+0000","last_peered":"2024-02-07T13:34:38.354764+0000","last_clean":"2024-02-07T13:34:38.354764+0000","last_became_active":"2024-02-07T13:34:15.436146+0000","last_became_peered":"2024-02-07T13:34:15.436146+0000","last_unstale":"2024-02-07T13:34:38.354764+0000","last_undegraded":"2024-02-07T13:34:38.354764+0000","last_fullsized":"2024-02-07T13:34:38.354764+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:44:33.250516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355136+0000","last_change":"2024-02-07T13:34:17.448268+0000","last_active":"2024-02-07T13:34:38.355136+0000","last_peered":"2024-02-07T13:34:38.355136+0000","last_clean":"2024-02-07T13:34:38.355136+0000","last_became_active":"2024-02-07T13:34:17.447973+0000","last_became_peered":"2024-02-07T13:34:17.447973+0000","last_unstale":"2024-02-07T13:34:38.355136+0000","last_undegraded":"2024-02-07T13:34:38.355136+0000","last_fullsized":"2024-02-07T13:34:38.355136+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:48:49.028545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350213+0000","last_change":"2024-02-07T13:34:13.429105+0000","last_active":"2024-02-07T13:34:38.350213+0000","last_peered":"2024-02-07T13:34:38.350213+0000","last_clean":"2024-02-07T13:34:38.350213+0000","last_became_active":"2024-02-07T13:34:13.428463+0000","last_became_peered":"2024-02-07T13:34:13.428463+0000","last_unstale":"2024-02-07T13:34:38.350213+0000","last_undegraded":"2024-02-07T13:34:38.350213+0000","last_fullsized":"2024-02-07T13:34:38.350213+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:28:06.020866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353581+0000","last_change":"2024-02-07T13:34:11.419837+0000","last_active":"2024-02-07T13:34:38.353581+0000","last_peered":"2024-02-07T13:34:38.353581+0000","last_clean":"2024-02-07T13:34:38.353581+0000","last_became_active":"2024-02-07T13:34:11.419461+0000","last_became_peered":"2024-02-07T13:34:11.419461+0000","last_unstale":"2024-02-07T13:34:38.353581+0000","last_undegraded":"2024-02-07T13:34:38.353581+0000","last_fullsized":"2024-02-07T13:34:38.353581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:53:54.714546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:40.453295+0000","last_change":"2024-02-07T13:34:00.029303+0000","last_active":"2024-02-07T13:34:40.453295+0000","last_peered":"2024-02-07T13:34:40.453295+0000","last_clean":"2024-02-07T13:34:40.453295+0000","last_became_active":"2024-02-07T13:34:00.028931+0000","last_became_peered":"2024-02-07T13:34:00.028931+0000","last_unstale":"2024-02-07T13:34:40.453295+0000","last_undegraded":"2024-02-07T13:34:40.453295+0000","last_fullsized":"2024-02-07T13:34:40.453295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:33:59.019138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:33:59.019138+0000","last_clean_scrub_stamp":"2024-02-07T13:33:59.019138+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:00:38.266935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353264+0000","last_change":"2024-02-07T13:34:15.437382+0000","last_active":"2024-02-07T13:34:38.353264+0000","last_peered":"2024-02-07T13:34:38.353264+0000","last_clean":"2024-02-07T13:34:38.353264+0000","last_became_active":"2024-02-07T13:34:15.436988+0000","last_became_peered":"2024-02-07T13:34:15.436988+0000","last_unstale":"2024-02-07T13:34:38.353264+0000","last_undegraded":"2024-02-07T13:34:38.353264+0000","last_fullsized":"2024-02-07T13:34:38.353264+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:10:23.455203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353783+0000","last_change":"2024-02-07T13:34:17.448578+0000","last_active":"2024-02-07T13:34:38.353783+0000","last_peered":"2024-02-07T13:34:38.353783+0000","last_clean":"2024-02-07T13:34:38.353783+0000","last_became_active":"2024-02-07T13:34:17.448120+0000","last_became_peered":"2024-02-07T13:34:17.448120+0000","last_unstale":"2024-02-07T13:34:38.353783+0000","last_undegraded":"2024-02-07T13:34:38.353783+0000","last_fullsized":"2024-02-07T13:34:38.353783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:35:35.025820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350702+0000","last_change":"2024-02-07T13:34:13.427655+0000","last_active":"2024-02-07T13:34:38.350702+0000","last_peered":"2024-02-07T13:34:38.350702+0000","last_clean":"2024-02-07T13:34:38.350702+0000","last_became_active":"2024-02-07T13:34:13.427289+0000","last_became_peered":"2024-02-07T13:34:13.427289+0000","last_unstale":"2024-02-07T13:34:38.350702+0000","last_undegraded":"2024-02-07T13:34:38.350702+0000","last_fullsized":"2024-02-07T13:34:38.350702+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:22:15.562439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352248+0000","last_change":"2024-02-07T13:34:11.425397+0000","last_active":"2024-02-07T13:34:38.352248+0000","last_peered":"2024-02-07T13:34:38.352248+0000","last_clean":"2024-02-07T13:34:38.352248+0000","last_became_active":"2024-02-07T13:34:11.424195+0000","last_became_peered":"2024-02-07T13:34:11.424195+0000","last_unstale":"2024-02-07T13:34:38.352248+0000","last_undegraded":"2024-02-07T13:34:38.352248+0000","last_fullsized":"2024-02-07T13:34:38.352248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:42:06.609380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357041+0000","last_change":"2024-02-07T13:34:15.431879+0000","last_active":"2024-02-07T13:34:38.357041+0000","last_peered":"2024-02-07T13:34:38.357041+0000","last_clean":"2024-02-07T13:34:38.357041+0000","last_became_active":"2024-02-07T13:34:15.431472+0000","last_became_peered":"2024-02-07T13:34:15.431472+0000","last_unstale":"2024-02-07T13:34:38.357041+0000","last_undegraded":"2024-02-07T13:34:38.357041+0000","last_fullsized":"2024-02-07T13:34:38.357041+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:32:45.940185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352759+0000","last_change":"2024-02-07T13:34:17.445806+0000","last_active":"2024-02-07T13:34:38.352759+0000","last_peered":"2024-02-07T13:34:38.352759+0000","last_clean":"2024-02-07T13:34:38.352759+0000","last_became_active":"2024-02-07T13:34:17.444983+0000","last_became_peered":"2024-02-07T13:34:17.444983+0000","last_unstale":"2024-02-07T13:34:38.352759+0000","last_undegraded":"2024-02-07T13:34:38.352759+0000","last_fullsized":"2024-02-07T13:34:38.352759+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:10:29.068022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356882+0000","last_change":"2024-02-07T13:34:11.420574+0000","last_active":"2024-02-07T13:34:38.356882+0000","last_peered":"2024-02-07T13:34:38.356882+0000","last_clean":"2024-02-07T13:34:38.356882+0000","last_became_active":"2024-02-07T13:34:11.419823+0000","last_became_peered":"2024-02-07T13:34:11.419823+0000","last_unstale":"2024-02-07T13:34:38.356882+0000","last_undegraded":"2024-02-07T13:34:38.356882+0000","last_fullsized":"2024-02-07T13:34:38.356882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:17:21.983253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468050+0000","last_change":"2024-02-07T13:34:13.430034+0000","last_active":"2024-02-07T13:34:41.468050+0000","last_peered":"2024-02-07T13:34:41.468050+0000","last_clean":"2024-02-07T13:34:41.468050+0000","last_became_active":"2024-02-07T13:34:13.429299+0000","last_became_peered":"2024-02-07T13:34:13.429299+0000","last_unstale":"2024-02-07T13:34:41.468050+0000","last_undegraded":"2024-02-07T13:34:41.468050+0000","last_fullsized":"2024-02-07T13:34:41.468050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:14:03.010029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468329+0000","last_change":"2024-02-07T13:34:15.431210+0000","last_active":"2024-02-07T13:34:41.468329+0000","last_peered":"2024-02-07T13:34:41.468329+0000","last_clean":"2024-02-07T13:34:41.468329+0000","last_became_active":"2024-02-07T13:34:15.430915+0000","last_became_peered":"2024-02-07T13:34:15.430915+0000","last_unstale":"2024-02-07T13:34:41.468329+0000","last_undegraded":"2024-02-07T13:34:41.468329+0000","last_fullsized":"2024-02-07T13:34:41.468329+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:08:19.151911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352151+0000","last_change":"2024-02-07T13:34:17.445783+0000","last_active":"2024-02-07T13:34:38.352151+0000","last_peered":"2024-02-07T13:34:38.352151+0000","last_clean":"2024-02-07T13:34:38.352151+0000","last_became_active":"2024-02-07T13:34:17.445059+0000","last_became_peered":"2024-02-07T13:34:17.445059+0000","last_unstale":"2024-02-07T13:34:38.352151+0000","last_undegraded":"2024-02-07T13:34:38.352151+0000","last_fullsized":"2024-02-07T13:34:38.352151+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:53:23.596324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353242+0000","last_change":"2024-02-07T13:34:11.420856+0000","last_active":"2024-02-07T13:34:38.353242+0000","last_peered":"2024-02-07T13:34:38.353242+0000","last_clean":"2024-02-07T13:34:38.353242+0000","last_became_active":"2024-02-07T13:34:11.419976+0000","last_became_peered":"2024-02-07T13:34:11.419976+0000","last_unstale":"2024-02-07T13:34:38.353242+0000","last_undegraded":"2024-02-07T13:34:38.353242+0000","last_fullsized":"2024-02-07T13:34:38.353242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:50:24.307168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354902+0000","last_change":"2024-02-07T13:34:13.429770+0000","last_active":"2024-02-07T13:34:38.354902+0000","last_peered":"2024-02-07T13:34:38.354902+0000","last_clean":"2024-02-07T13:34:38.354902+0000","last_became_active":"2024-02-07T13:34:13.428916+0000","last_became_peered":"2024-02-07T13:34:13.428916+0000","last_unstale":"2024-02-07T13:34:38.354902+0000","last_undegraded":"2024-02-07T13:34:38.354902+0000","last_fullsized":"2024-02-07T13:34:38.354902+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:46:28.881402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468048+0000","last_change":"2024-02-07T13:34:15.436995+0000","last_active":"2024-02-07T13:34:41.468048+0000","last_peered":"2024-02-07T13:34:41.468048+0000","last_clean":"2024-02-07T13:34:41.468048+0000","last_became_active":"2024-02-07T13:34:15.436634+0000","last_became_peered":"2024-02-07T13:34:15.436634+0000","last_unstale":"2024-02-07T13:34:41.468048+0000","last_undegraded":"2024-02-07T13:34:41.468048+0000","last_fullsized":"2024-02-07T13:34:41.468048+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:33:35.094693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355698+0000","last_change":"2024-02-07T13:34:17.447846+0000","last_active":"2024-02-07T13:34:38.355698+0000","last_peered":"2024-02-07T13:34:38.355698+0000","last_clean":"2024-02-07T13:34:38.355698+0000","last_became_active":"2024-02-07T13:34:17.447523+0000","last_became_peered":"2024-02-07T13:34:17.447523+0000","last_unstale":"2024-02-07T13:34:38.355698+0000","last_undegraded":"2024-02-07T13:34:38.355698+0000","last_fullsized":"2024-02-07T13:34:38.355698+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:03:39.234809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355257+0000","last_change":"2024-02-07T13:34:11.424004+0000","last_active":"2024-02-07T13:34:38.355257+0000","last_peered":"2024-02-07T13:34:38.355257+0000","last_clean":"2024-02-07T13:34:38.355257+0000","last_became_active":"2024-02-07T13:34:11.423635+0000","last_became_peered":"2024-02-07T13:34:11.423635+0000","last_unstale":"2024-02-07T13:34:38.355257+0000","last_undegraded":"2024-02-07T13:34:38.355257+0000","last_fullsized":"2024-02-07T13:34:38.355257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:09:16.066441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353048+0000","last_change":"2024-02-07T13:34:13.430111+0000","last_active":"2024-02-07T13:34:38.353048+0000","last_peered":"2024-02-07T13:34:38.353048+0000","last_clean":"2024-02-07T13:34:38.353048+0000","last_became_active":"2024-02-07T13:34:13.429006+0000","last_became_peered":"2024-02-07T13:34:13.429006+0000","last_unstale":"2024-02-07T13:34:38.353048+0000","last_undegraded":"2024-02-07T13:34:38.353048+0000","last_fullsized":"2024-02-07T13:34:38.353048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:41:34.701748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354933+0000","last_change":"2024-02-07T13:34:15.432100+0000","last_active":"2024-02-07T13:34:38.354933+0000","last_peered":"2024-02-07T13:34:38.354933+0000","last_clean":"2024-02-07T13:34:38.354933+0000","last_became_active":"2024-02-07T13:34:15.431821+0000","last_became_peered":"2024-02-07T13:34:15.431821+0000","last_unstale":"2024-02-07T13:34:38.354933+0000","last_undegraded":"2024-02-07T13:34:38.354933+0000","last_fullsized":"2024-02-07T13:34:38.354933+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:04:15.823910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353904+0000","last_change":"2024-02-07T13:34:17.447432+0000","last_active":"2024-02-07T13:34:38.353904+0000","last_peered":"2024-02-07T13:34:38.353904+0000","last_clean":"2024-02-07T13:34:38.353904+0000","last_became_active":"2024-02-07T13:34:17.447023+0000","last_became_peered":"2024-02-07T13:34:17.447023+0000","last_unstale":"2024-02-07T13:34:38.353904+0000","last_undegraded":"2024-02-07T13:34:38.353904+0000","last_fullsized":"2024-02-07T13:34:38.353904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:34:22.049166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468116+0000","last_change":"2024-02-07T13:34:15.437241+0000","last_active":"2024-02-07T13:34:41.468116+0000","last_peered":"2024-02-07T13:34:41.468116+0000","last_clean":"2024-02-07T13:34:41.468116+0000","last_became_active":"2024-02-07T13:34:15.436551+0000","last_became_peered":"2024-02-07T13:34:15.436551+0000","last_unstale":"2024-02-07T13:34:41.468116+0000","last_undegraded":"2024-02-07T13:34:41.468116+0000","last_fullsized":"2024-02-07T13:34:41.468116+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:44:13.780993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352059+0000","last_change":"2024-02-07T13:34:11.425684+0000","last_active":"2024-02-07T13:34:38.352059+0000","last_peered":"2024-02-07T13:34:38.352059+0000","last_clean":"2024-02-07T13:34:38.352059+0000","last_became_active":"2024-02-07T13:34:11.425405+0000","last_became_peered":"2024-02-07T13:34:11.425405+0000","last_unstale":"2024-02-07T13:34:38.352059+0000","last_undegraded":"2024-02-07T13:34:38.352059+0000","last_fullsized":"2024-02-07T13:34:38.352059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:21:53.365086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355412+0000","last_change":"2024-02-07T13:34:13.435291+0000","last_active":"2024-02-07T13:34:38.355412+0000","last_peered":"2024-02-07T13:34:38.355412+0000","last_clean":"2024-02-07T13:34:38.355412+0000","last_became_active":"2024-02-07T13:34:13.434664+0000","last_became_peered":"2024-02-07T13:34:13.434664+0000","last_unstale":"2024-02-07T13:34:38.355412+0000","last_undegraded":"2024-02-07T13:34:38.355412+0000","last_fullsized":"2024-02-07T13:34:38.355412+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:45:38.841078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357124+0000","last_change":"2024-02-07T13:34:17.450157+0000","last_active":"2024-02-07T13:34:38.357124+0000","last_peered":"2024-02-07T13:34:38.357124+0000","last_clean":"2024-02-07T13:34:38.357124+0000","last_became_active":"2024-02-07T13:34:17.449576+0000","last_became_peered":"2024-02-07T13:34:17.449576+0000","last_unstale":"2024-02-07T13:34:38.357124+0000","last_undegraded":"2024-02-07T13:34:38.357124+0000","last_fullsized":"2024-02-07T13:34:38.357124+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:05:31.746195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353330+0000","last_change":"2024-02-07T13:34:15.436971+0000","last_active":"2024-02-07T13:34:38.353330+0000","last_peered":"2024-02-07T13:34:38.353330+0000","last_clean":"2024-02-07T13:34:38.353330+0000","last_became_active":"2024-02-07T13:34:15.436451+0000","last_became_peered":"2024-02-07T13:34:15.436451+0000","last_unstale":"2024-02-07T13:34:38.353330+0000","last_undegraded":"2024-02-07T13:34:38.353330+0000","last_fullsized":"2024-02-07T13:34:38.353330+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:08:21.213044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350877+0000","last_change":"2024-02-07T13:34:11.420848+0000","last_active":"2024-02-07T13:34:38.350877+0000","last_peered":"2024-02-07T13:34:38.350877+0000","last_clean":"2024-02-07T13:34:38.350877+0000","last_became_active":"2024-02-07T13:34:11.419695+0000","last_became_peered":"2024-02-07T13:34:11.419695+0000","last_unstale":"2024-02-07T13:34:38.350877+0000","last_undegraded":"2024-02-07T13:34:38.350877+0000","last_fullsized":"2024-02-07T13:34:38.350877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:39:49.334893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355152+0000","last_change":"2024-02-07T13:34:13.432434+0000","last_active":"2024-02-07T13:34:38.355152+0000","last_peered":"2024-02-07T13:34:38.355152+0000","last_clean":"2024-02-07T13:34:38.355152+0000","last_became_active":"2024-02-07T13:34:13.431810+0000","last_became_peered":"2024-02-07T13:34:13.431810+0000","last_unstale":"2024-02-07T13:34:38.355152+0000","last_undegraded":"2024-02-07T13:34:38.355152+0000","last_fullsized":"2024-02-07T13:34:38.355152+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:41:44.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356156+0000","last_change":"2024-02-07T13:34:17.449293+0000","last_active":"2024-02-07T13:34:38.356156+0000","last_peered":"2024-02-07T13:34:38.356156+0000","last_clean":"2024-02-07T13:34:38.356156+0000","last_became_active":"2024-02-07T13:34:17.449004+0000","last_became_peered":"2024-02-07T13:34:17.449004+0000","last_unstale":"2024-02-07T13:34:38.356156+0000","last_undegraded":"2024-02-07T13:34:38.356156+0000","last_fullsized":"2024-02-07T13:34:38.356156+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:46:45.518482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355986+0000","last_change":"2024-02-07T13:34:11.419894+0000","last_active":"2024-02-07T13:34:38.355986+0000","last_peered":"2024-02-07T13:34:38.355986+0000","last_clean":"2024-02-07T13:34:38.355986+0000","last_became_active":"2024-02-07T13:34:11.419240+0000","last_became_peered":"2024-02-07T13:34:11.419240+0000","last_unstale":"2024-02-07T13:34:38.355986+0000","last_undegraded":"2024-02-07T13:34:38.355986+0000","last_fullsized":"2024-02-07T13:34:38.355986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:47:26.420102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352123+0000","last_change":"2024-02-07T13:34:13.427521+0000","last_active":"2024-02-07T13:34:38.352123+0000","last_peered":"2024-02-07T13:34:38.352123+0000","last_clean":"2024-02-07T13:34:38.352123+0000","last_became_active":"2024-02-07T13:34:13.427180+0000","last_became_peered":"2024-02-07T13:34:13.427180+0000","last_unstale":"2024-02-07T13:34:38.352123+0000","last_undegraded":"2024-02-07T13:34:38.352123+0000","last_fullsized":"2024-02-07T13:34:38.352123+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:47:00.901391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354966+0000","last_change":"2024-02-07T13:34:15.438867+0000","last_active":"2024-02-07T13:34:38.354966+0000","last_peered":"2024-02-07T13:34:38.354966+0000","last_clean":"2024-02-07T13:34:38.354966+0000","last_became_active":"2024-02-07T13:34:15.438004+0000","last_became_peered":"2024-02-07T13:34:15.438004+0000","last_unstale":"2024-02-07T13:34:38.354966+0000","last_undegraded":"2024-02-07T13:34:38.354966+0000","last_fullsized":"2024-02-07T13:34:38.354966+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:55:45.191362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350484+0000","last_change":"2024-02-07T13:34:17.446334+0000","last_active":"2024-02-07T13:34:38.350484+0000","last_peered":"2024-02-07T13:34:38.350484+0000","last_clean":"2024-02-07T13:34:38.350484+0000","last_became_active":"2024-02-07T13:34:17.445998+0000","last_became_peered":"2024-02-07T13:34:17.445998+0000","last_unstale":"2024-02-07T13:34:38.350484+0000","last_undegraded":"2024-02-07T13:34:38.350484+0000","last_fullsized":"2024-02-07T13:34:38.350484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:50:59.668742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:41.468115+0000","last_change":"2024-02-07T13:34:15.437443+0000","last_active":"2024-02-07T13:34:41.468115+0000","last_peered":"2024-02-07T13:34:41.468115+0000","last_clean":"2024-02-07T13:34:41.468115+0000","last_became_active":"2024-02-07T13:34:15.436781+0000","last_became_peered":"2024-02-07T13:34:15.436781+0000","last_unstale":"2024-02-07T13:34:41.468115+0000","last_undegraded":"2024-02-07T13:34:41.468115+0000","last_fullsized":"2024-02-07T13:34:41.468115+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:16:49.601811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356242+0000","last_change":"2024-02-07T13:34:11.420420+0000","last_active":"2024-02-07T13:34:38.356242+0000","last_peered":"2024-02-07T13:34:38.356242+0000","last_clean":"2024-02-07T13:34:38.356242+0000","last_became_active":"2024-02-07T13:34:11.419430+0000","last_became_peered":"2024-02-07T13:34:11.419430+0000","last_unstale":"2024-02-07T13:34:38.356242+0000","last_undegraded":"2024-02-07T13:34:38.356242+0000","last_fullsized":"2024-02-07T13:34:38.356242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:40:47.899859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352658+0000","last_change":"2024-02-07T13:34:13.429252+0000","last_active":"2024-02-07T13:34:38.352658+0000","last_peered":"2024-02-07T13:34:38.352658+0000","last_clean":"2024-02-07T13:34:38.352658+0000","last_became_active":"2024-02-07T13:34:13.428462+0000","last_became_peered":"2024-02-07T13:34:13.428462+0000","last_unstale":"2024-02-07T13:34:38.352658+0000","last_undegraded":"2024-02-07T13:34:38.352658+0000","last_fullsized":"2024-02-07T13:34:38.352658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:36:31.457455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355461+0000","last_change":"2024-02-07T13:34:17.448542+0000","last_active":"2024-02-07T13:34:38.355461+0000","last_peered":"2024-02-07T13:34:38.355461+0000","last_clean":"2024-02-07T13:34:38.355461+0000","last_became_active":"2024-02-07T13:34:17.447650+0000","last_became_peered":"2024-02-07T13:34:17.447650+0000","last_unstale":"2024-02-07T13:34:38.355461+0000","last_undegraded":"2024-02-07T13:34:38.355461+0000","last_fullsized":"2024-02-07T13:34:38.355461+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:44:11.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355749+0000","last_change":"2024-02-07T13:34:11.424766+0000","last_active":"2024-02-07T13:34:38.355749+0000","last_peered":"2024-02-07T13:34:38.355749+0000","last_clean":"2024-02-07T13:34:38.355749+0000","last_became_active":"2024-02-07T13:34:11.424183+0000","last_became_peered":"2024-02-07T13:34:11.424183+0000","last_unstale":"2024-02-07T13:34:38.355749+0000","last_undegraded":"2024-02-07T13:34:38.355749+0000","last_fullsized":"2024-02-07T13:34:38.355749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:12:57.668280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353394+0000","last_change":"2024-02-07T13:34:13.429976+0000","last_active":"2024-02-07T13:34:38.353394+0000","last_peered":"2024-02-07T13:34:38.353394+0000","last_clean":"2024-02-07T13:34:38.353394+0000","last_became_active":"2024-02-07T13:34:13.428767+0000","last_became_peered":"2024-02-07T13:34:13.428767+0000","last_unstale":"2024-02-07T13:34:38.353394+0000","last_undegraded":"2024-02-07T13:34:38.353394+0000","last_fullsized":"2024-02-07T13:34:38.353394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:39:49.240386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352504+0000","last_change":"2024-02-07T13:34:15.437645+0000","last_active":"2024-02-07T13:34:38.352504+0000","last_peered":"2024-02-07T13:34:38.352504+0000","last_clean":"2024-02-07T13:34:38.352504+0000","last_became_active":"2024-02-07T13:34:15.436692+0000","last_became_peered":"2024-02-07T13:34:15.436692+0000","last_unstale":"2024-02-07T13:34:38.352504+0000","last_undegraded":"2024-02-07T13:34:38.352504+0000","last_fullsized":"2024-02-07T13:34:38.352504+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:06:56.470469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.349799+0000","last_change":"2024-02-07T13:34:17.444188+0000","last_active":"2024-02-07T13:34:38.349799+0000","last_peered":"2024-02-07T13:34:38.349799+0000","last_clean":"2024-02-07T13:34:38.349799+0000","last_became_active":"2024-02-07T13:34:17.443808+0000","last_became_peered":"2024-02-07T13:34:17.443808+0000","last_unstale":"2024-02-07T13:34:38.349799+0000","last_undegraded":"2024-02-07T13:34:38.349799+0000","last_fullsized":"2024-02-07T13:34:38.349799+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:30:40.258989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356650+0000","last_change":"2024-02-07T13:34:11.423931+0000","last_active":"2024-02-07T13:34:38.356650+0000","last_peered":"2024-02-07T13:34:38.356650+0000","last_clean":"2024-02-07T13:34:38.356650+0000","last_became_active":"2024-02-07T13:34:11.423643+0000","last_became_peered":"2024-02-07T13:34:11.423643+0000","last_unstale":"2024-02-07T13:34:38.356650+0000","last_undegraded":"2024-02-07T13:34:38.356650+0000","last_fullsized":"2024-02-07T13:34:38.356650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:15:21.761202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355134+0000","last_change":"2024-02-07T13:34:13.428659+0000","last_active":"2024-02-07T13:34:38.355134+0000","last_peered":"2024-02-07T13:34:38.355134+0000","last_clean":"2024-02-07T13:34:38.355134+0000","last_became_active":"2024-02-07T13:34:13.427422+0000","last_became_peered":"2024-02-07T13:34:13.427422+0000","last_unstale":"2024-02-07T13:34:38.355134+0000","last_undegraded":"2024-02-07T13:34:38.355134+0000","last_fullsized":"2024-02-07T13:34:38.355134+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:32:25.128929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350520+0000","last_change":"2024-02-07T13:34:15.435366+0000","last_active":"2024-02-07T13:34:38.350520+0000","last_peered":"2024-02-07T13:34:38.350520+0000","last_clean":"2024-02-07T13:34:38.350520+0000","last_became_active":"2024-02-07T13:34:15.435045+0000","last_became_peered":"2024-02-07T13:34:15.435045+0000","last_unstale":"2024-02-07T13:34:38.350520+0000","last_undegraded":"2024-02-07T13:34:38.350520+0000","last_fullsized":"2024-02-07T13:34:38.350520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:50:15.285728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352030+0000","last_change":"2024-02-07T13:34:17.447321+0000","last_active":"2024-02-07T13:34:38.352030+0000","last_peered":"2024-02-07T13:34:38.352030+0000","last_clean":"2024-02-07T13:34:38.352030+0000","last_became_active":"2024-02-07T13:34:17.446988+0000","last_became_peered":"2024-02-07T13:34:17.446988+0000","last_unstale":"2024-02-07T13:34:38.352030+0000","last_undegraded":"2024-02-07T13:34:38.352030+0000","last_fullsized":"2024-02-07T13:34:38.352030+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:43:01.522028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352957+0000","last_change":"2024-02-07T13:34:11.420939+0000","last_active":"2024-02-07T13:34:38.352957+0000","last_peered":"2024-02-07T13:34:38.352957+0000","last_clean":"2024-02-07T13:34:38.352957+0000","last_became_active":"2024-02-07T13:34:11.420072+0000","last_became_peered":"2024-02-07T13:34:11.420072+0000","last_unstale":"2024-02-07T13:34:38.352957+0000","last_undegraded":"2024-02-07T13:34:38.352957+0000","last_fullsized":"2024-02-07T13:34:38.352957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:22:54.546023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354737+0000","last_change":"2024-02-07T13:34:13.430137+0000","last_active":"2024-02-07T13:34:38.354737+0000","last_peered":"2024-02-07T13:34:38.354737+0000","last_clean":"2024-02-07T13:34:38.354737+0000","last_became_active":"2024-02-07T13:34:13.429300+0000","last_became_peered":"2024-02-07T13:34:13.429300+0000","last_unstale":"2024-02-07T13:34:38.354737+0000","last_undegraded":"2024-02-07T13:34:38.354737+0000","last_fullsized":"2024-02-07T13:34:38.354737+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:05:04.356701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355380+0000","last_change":"2024-02-07T13:34:15.436747+0000","last_active":"2024-02-07T13:34:38.355380+0000","last_peered":"2024-02-07T13:34:38.355380+0000","last_clean":"2024-02-07T13:34:38.355380+0000","last_became_active":"2024-02-07T13:34:15.436486+0000","last_became_peered":"2024-02-07T13:34:15.436486+0000","last_unstale":"2024-02-07T13:34:38.355380+0000","last_undegraded":"2024-02-07T13:34:38.355380+0000","last_fullsized":"2024-02-07T13:34:38.355380+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:08:42.751142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411341,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738391,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93704692,"statfs":{"total":95990841344,"available":95953604608,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 7 13:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.499},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 7 13:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-07T13:34:54.222 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-07T13:34:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:54 smithi094 ceph-mon[25576]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:34:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:54 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:34:54.479 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:34:54.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:54 smithi190 ceph-mon[30713]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:34:54.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:54 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:34:54.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:54 smithi138 ceph-mon[30535]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:34:54.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:54 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:34:55.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:55 smithi094 ceph-mon[25576]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:34:55.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:55 smithi190 ceph-mon[30713]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:34:55.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:55 smithi138 ceph-mon[30535]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:34:56.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:56 smithi094 ceph-mon[25576]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:34:56.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:56 smithi190 ceph-mon[30713]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:34:56.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:56 smithi138 ceph-mon[30535]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:34:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:34:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-02-07T13:34:58.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:58 smithi094 ceph-mon[25576]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:34:58.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:58 smithi190 ceph-mon[30713]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:34:58.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:58 smithi138 ceph-mon[30535]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:34:58.815 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-02-07T13:34:56.363180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":100772,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":97659,"kb_avail":281122396,"statfs":{"total":287972524032,"available":287869333504,"internally_reserved":0,"allocated":3158016,"data_stored":2100177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":100003151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002559"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352694+0000","last_change":"2024-02-07T13:34:17.446711+0000","last_active":"2024-02-07T13:34:38.352694+0000","last_peered":"2024-02-07T13:34:38.352694+0000","last_clean":"2024-02-07T13:34:38.352694+0000","last_became_active":"2024-02-07T13:34:17.446312+0000","last_became_peered":"2024-02-07T13:34:17.446312+0000","last_unstale":"2024-02-07T13:34:38.352694+0000","last_undegraded":"2024-02-07T13:34:38.352694+0000","last_fullsized":"2024-02-07T13:34:38.352694+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:00:36.521357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354840+0000","last_change":"2024-02-07T13:34:15.432414+0000","last_active":"2024-02-07T13:34:38.354840+0000","last_peered":"2024-02-07T13:34:38.354840+0000","last_clean":"2024-02-07T13:34:38.354840+0000","last_became_active":"2024-02-07T13:34:15.432114+0000","last_became_peered":"2024-02-07T13:34:15.432114+0000","last_unstale":"2024-02-07T13:34:38.354840+0000","last_undegraded":"2024-02-07T13:34:38.354840+0000","last_fullsized":"2024-02-07T13:34:38.354840+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:42:13.628394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353543+0000","last_change":"2024-02-07T13:34:11.419834+0000","last_active":"2024-02-07T13:34:38.353543+0000","last_peered":"2024-02-07T13:34:38.353543+0000","last_clean":"2024-02-07T13:34:38.353543+0000","last_became_active":"2024-02-07T13:34:11.419488+0000","last_became_peered":"2024-02-07T13:34:11.419488+0000","last_unstale":"2024-02-07T13:34:38.353543+0000","last_undegraded":"2024-02-07T13:34:38.353543+0000","last_fullsized":"2024-02-07T13:34:38.353543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:29:52.258771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352836+0000","last_change":"2024-02-07T13:34:13.428893+0000","last_active":"2024-02-07T13:34:38.352836+0000","last_peered":"2024-02-07T13:34:38.352836+0000","last_clean":"2024-02-07T13:34:38.352836+0000","last_became_active":"2024-02-07T13:34:13.428518+0000","last_became_peered":"2024-02-07T13:34:13.428518+0000","last_unstale":"2024-02-07T13:34:38.352836+0000","last_undegraded":"2024-02-07T13:34:38.352836+0000","last_fullsized":"2024-02-07T13:34:38.352836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:20:01.615410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350135+0000","last_change":"2024-02-07T13:34:17.444471+0000","last_active":"2024-02-07T13:34:38.350135+0000","last_peered":"2024-02-07T13:34:38.350135+0000","last_clean":"2024-02-07T13:34:38.350135+0000","last_became_active":"2024-02-07T13:34:17.443947+0000","last_became_peered":"2024-02-07T13:34:17.443947+0000","last_unstale":"2024-02-07T13:34:38.350135+0000","last_undegraded":"2024-02-07T13:34:38.350135+0000","last_fullsized":"2024-02-07T13:34:38.350135+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:53:32.450578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352424+0000","last_change":"2024-02-07T13:34:15.439337+0000","last_active":"2024-02-07T13:34:38.352424+0000","last_peered":"2024-02-07T13:34:38.352424+0000","last_clean":"2024-02-07T13:34:38.352424+0000","last_became_active":"2024-02-07T13:34:15.439001+0000","last_became_peered":"2024-02-07T13:34:15.439001+0000","last_unstale":"2024-02-07T13:34:38.352424+0000","last_undegraded":"2024-02-07T13:34:38.352424+0000","last_fullsized":"2024-02-07T13:34:38.352424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:13:18.996828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353395+0000","last_change":"2024-02-07T13:34:11.420536+0000","last_active":"2024-02-07T13:34:38.353395+0000","last_peered":"2024-02-07T13:34:38.353395+0000","last_clean":"2024-02-07T13:34:38.353395+0000","last_became_active":"2024-02-07T13:34:11.419608+0000","last_became_peered":"2024-02-07T13:34:11.419608+0000","last_unstale":"2024-02-07T13:34:38.353395+0000","last_undegraded":"2024-02-07T13:34:38.353395+0000","last_fullsized":"2024-02-07T13:34:38.353395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:01:57.720890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353413+0000","last_change":"2024-02-07T13:34:13.429132+0000","last_active":"2024-02-07T13:34:38.353413+0000","last_peered":"2024-02-07T13:34:38.353413+0000","last_clean":"2024-02-07T13:34:38.353413+0000","last_became_active":"2024-02-07T13:34:13.428787+0000","last_became_peered":"2024-02-07T13:34:13.428787+0000","last_unstale":"2024-02-07T13:34:38.353413+0000","last_undegraded":"2024-02-07T13:34:38.353413+0000","last_fullsized":"2024-02-07T13:34:38.353413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:59:50.143822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355262+0000","last_change":"2024-02-07T13:34:17.448969+0000","last_active":"2024-02-07T13:34:38.355262+0000","last_peered":"2024-02-07T13:34:38.355262+0000","last_clean":"2024-02-07T13:34:38.355262+0000","last_became_active":"2024-02-07T13:34:17.448042+0000","last_became_peered":"2024-02-07T13:34:17.448042+0000","last_unstale":"2024-02-07T13:34:38.355262+0000","last_undegraded":"2024-02-07T13:34:38.355262+0000","last_fullsized":"2024-02-07T13:34:38.355262+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:57:59.770390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352686+0000","last_change":"2024-02-07T13:34:15.439656+0000","last_active":"2024-02-07T13:34:38.352686+0000","last_peered":"2024-02-07T13:34:38.352686+0000","last_clean":"2024-02-07T13:34:38.352686+0000","last_became_active":"2024-02-07T13:34:15.439243+0000","last_became_peered":"2024-02-07T13:34:15.439243+0000","last_unstale":"2024-02-07T13:34:38.352686+0000","last_undegraded":"2024-02-07T13:34:38.352686+0000","last_fullsized":"2024-02-07T13:34:38.352686+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:53:07.879042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357094+0000","last_change":"2024-02-07T13:34:11.420151+0000","last_active":"2024-02-07T13:34:38.357094+0000","last_peered":"2024-02-07T13:34:38.357094+0000","last_clean":"2024-02-07T13:34:38.357094+0000","last_became_active":"2024-02-07T13:34:11.419334+0000","last_became_peered":"2024-02-07T13:34:11.419334+0000","last_unstale":"2024-02-07T13:34:38.357094+0000","last_undegraded":"2024-02-07T13:34:38.357094+0000","last_fullsized":"2024-02-07T13:34:38.357094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:52:37.194909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355328+0000","last_change":"2024-02-07T13:34:13.430098+0000","last_active":"2024-02-07T13:34:38.355328+0000","last_peered":"2024-02-07T13:34:38.355328+0000","last_clean":"2024-02-07T13:34:38.355328+0000","last_became_active":"2024-02-07T13:34:13.429726+0000","last_became_peered":"2024-02-07T13:34:13.429726+0000","last_unstale":"2024-02-07T13:34:38.355328+0000","last_undegraded":"2024-02-07T13:34:38.355328+0000","last_fullsized":"2024-02-07T13:34:38.355328+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:07:05.787118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353005+0000","last_change":"2024-02-07T13:34:17.445312+0000","last_active":"2024-02-07T13:34:38.353005+0000","last_peered":"2024-02-07T13:34:38.353005+0000","last_clean":"2024-02-07T13:34:38.353005+0000","last_became_active":"2024-02-07T13:34:17.444746+0000","last_became_peered":"2024-02-07T13:34:17.444746+0000","last_unstale":"2024-02-07T13:34:38.353005+0000","last_undegraded":"2024-02-07T13:34:38.353005+0000","last_fullsized":"2024-02-07T13:34:38.353005+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:41:51.995066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352344+0000","last_change":"2024-02-07T13:34:15.437222+0000","last_active":"2024-02-07T13:34:38.352344+0000","last_peered":"2024-02-07T13:34:38.352344+0000","last_clean":"2024-02-07T13:34:38.352344+0000","last_became_active":"2024-02-07T13:34:15.436816+0000","last_became_peered":"2024-02-07T13:34:15.436816+0000","last_unstale":"2024-02-07T13:34:38.352344+0000","last_undegraded":"2024-02-07T13:34:38.352344+0000","last_fullsized":"2024-02-07T13:34:38.352344+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:41:20.282903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355586+0000","last_change":"2024-02-07T13:34:11.423934+0000","last_active":"2024-02-07T13:34:38.355586+0000","last_peered":"2024-02-07T13:34:38.355586+0000","last_clean":"2024-02-07T13:34:38.355586+0000","last_became_active":"2024-02-07T13:34:11.423615+0000","last_became_peered":"2024-02-07T13:34:11.423615+0000","last_unstale":"2024-02-07T13:34:38.355586+0000","last_undegraded":"2024-02-07T13:34:38.355586+0000","last_fullsized":"2024-02-07T13:34:38.355586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:53:53.398276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356984+0000","last_change":"2024-02-07T13:34:13.435374+0000","last_active":"2024-02-07T13:34:38.356984+0000","last_peered":"2024-02-07T13:34:38.356984+0000","last_clean":"2024-02-07T13:34:38.356984+0000","last_became_active":"2024-02-07T13:34:13.435086+0000","last_became_peered":"2024-02-07T13:34:13.435086+0000","last_unstale":"2024-02-07T13:34:38.356984+0000","last_undegraded":"2024-02-07T13:34:38.356984+0000","last_fullsized":"2024-02-07T13:34:38.356984+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:06:39.164361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354876+0000","last_change":"2024-02-07T13:34:17.447806+0000","last_active":"2024-02-07T13:34:38.354876+0000","last_peered":"2024-02-07T13:34:38.354876+0000","last_clean":"2024-02-07T13:34:38.354876+0000","last_became_active":"2024-02-07T13:34:17.446951+0000","last_became_peered":"2024-02-07T13:34:17.446951+0000","last_unstale":"2024-02-07T13:34:38.354876+0000","last_undegraded":"2024-02-07T13:34:38.354876+0000","last_fullsized":"2024-02-07T13:34:38.354876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:02:08.290401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356184+0000","last_change":"2024-02-07T13:34:15.433608+0000","last_active":"2024-02-07T13:34:38.356184+0000","last_peered":"2024-02-07T13:34:38.356184+0000","last_clean":"2024-02-07T13:34:38.356184+0000","last_became_active":"2024-02-07T13:34:15.432492+0000","last_became_peered":"2024-02-07T13:34:15.432492+0000","last_unstale":"2024-02-07T13:34:38.356184+0000","last_undegraded":"2024-02-07T13:34:38.356184+0000","last_fullsized":"2024-02-07T13:34:38.356184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:47:33.155614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356646+0000","last_change":"2024-02-07T13:34:11.420795+0000","last_active":"2024-02-07T13:34:38.356646+0000","last_peered":"2024-02-07T13:34:38.356646+0000","last_clean":"2024-02-07T13:34:38.356646+0000","last_became_active":"2024-02-07T13:34:11.420051+0000","last_became_peered":"2024-02-07T13:34:11.420051+0000","last_unstale":"2024-02-07T13:34:38.356646+0000","last_undegraded":"2024-02-07T13:34:38.356646+0000","last_fullsized":"2024-02-07T13:34:38.356646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:08:42.458805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355995+0000","last_change":"2024-02-07T13:34:13.435150+0000","last_active":"2024-02-07T13:34:38.355995+0000","last_peered":"2024-02-07T13:34:38.355995+0000","last_clean":"2024-02-07T13:34:38.355995+0000","last_became_active":"2024-02-07T13:34:13.434624+0000","last_became_peered":"2024-02-07T13:34:13.434624+0000","last_unstale":"2024-02-07T13:34:38.355995+0000","last_undegraded":"2024-02-07T13:34:38.355995+0000","last_fullsized":"2024-02-07T13:34:38.355995+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:22:30.337210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352252+0000","last_change":"2024-02-07T13:34:17.448906+0000","last_active":"2024-02-07T13:34:38.352252+0000","last_peered":"2024-02-07T13:34:38.352252+0000","last_clean":"2024-02-07T13:34:38.352252+0000","last_became_active":"2024-02-07T13:34:17.448536+0000","last_became_peered":"2024-02-07T13:34:17.448536+0000","last_unstale":"2024-02-07T13:34:38.352252+0000","last_undegraded":"2024-02-07T13:34:38.352252+0000","last_fullsized":"2024-02-07T13:34:38.352252+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:17:02.360141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355841+0000","last_change":"2024-02-07T13:34:15.432160+0000","last_active":"2024-02-07T13:34:38.355841+0000","last_peered":"2024-02-07T13:34:38.355841+0000","last_clean":"2024-02-07T13:34:38.355841+0000","last_became_active":"2024-02-07T13:34:15.431887+0000","last_became_peered":"2024-02-07T13:34:15.431887+0000","last_unstale":"2024-02-07T13:34:38.355841+0000","last_undegraded":"2024-02-07T13:34:38.355841+0000","last_fullsized":"2024-02-07T13:34:38.355841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:03:30.073273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352808+0000","last_change":"2024-02-07T13:34:11.419829+0000","last_active":"2024-02-07T13:34:38.352808+0000","last_peered":"2024-02-07T13:34:38.352808+0000","last_clean":"2024-02-07T13:34:38.352808+0000","last_became_active":"2024-02-07T13:34:11.419443+0000","last_became_peered":"2024-02-07T13:34:11.419443+0000","last_unstale":"2024-02-07T13:34:38.352808+0000","last_undegraded":"2024-02-07T13:34:38.352808+0000","last_fullsized":"2024-02-07T13:34:38.352808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:59:25.520579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356132+0000","last_change":"2024-02-07T13:34:13.432437+0000","last_active":"2024-02-07T13:34:38.356132+0000","last_peered":"2024-02-07T13:34:38.356132+0000","last_clean":"2024-02-07T13:34:38.356132+0000","last_became_active":"2024-02-07T13:34:13.431768+0000","last_became_peered":"2024-02-07T13:34:13.431768+0000","last_unstale":"2024-02-07T13:34:38.356132+0000","last_undegraded":"2024-02-07T13:34:38.356132+0000","last_fullsized":"2024-02-07T13:34:38.356132+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:03:28.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.351017+0000","last_change":"2024-02-07T13:34:17.449510+0000","last_active":"2024-02-07T13:34:38.351017+0000","last_peered":"2024-02-07T13:34:38.351017+0000","last_clean":"2024-02-07T13:34:38.351017+0000","last_became_active":"2024-02-07T13:34:17.449156+0000","last_became_peered":"2024-02-07T13:34:17.449156+0000","last_unstale":"2024-02-07T13:34:38.351017+0000","last_undegraded":"2024-02-07T13:34:38.351017+0000","last_fullsized":"2024-02-07T13:34:38.351017+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:50:37.678973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469625+0000","last_change":"2024-02-07T13:34:15.434396+0000","last_active":"2024-02-07T13:34:46.469625+0000","last_peered":"2024-02-07T13:34:46.469625+0000","last_clean":"2024-02-07T13:34:46.469625+0000","last_became_active":"2024-02-07T13:34:15.433652+0000","last_became_peered":"2024-02-07T13:34:15.433652+0000","last_unstale":"2024-02-07T13:34:46.469625+0000","last_undegraded":"2024-02-07T13:34:46.469625+0000","last_fullsized":"2024-02-07T13:34:46.469625+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:58:59.586215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350574+0000","last_change":"2024-02-07T13:34:11.420820+0000","last_active":"2024-02-07T13:34:38.350574+0000","last_peered":"2024-02-07T13:34:38.350574+0000","last_clean":"2024-02-07T13:34:38.350574+0000","last_became_active":"2024-02-07T13:34:11.420009+0000","last_became_peered":"2024-02-07T13:34:11.420009+0000","last_unstale":"2024-02-07T13:34:38.350574+0000","last_undegraded":"2024-02-07T13:34:38.350574+0000","last_fullsized":"2024-02-07T13:34:38.350574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:20:49.141845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":920,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.349981+0000","last_change":"2024-02-07T13:34:13.430135+0000","last_active":"2024-02-07T13:34:38.349981+0000","last_peered":"2024-02-07T13:34:38.349981+0000","last_clean":"2024-02-07T13:34:38.349981+0000","last_became_active":"2024-02-07T13:34:13.428963+0000","last_became_peered":"2024-02-07T13:34:13.428963+0000","last_unstale":"2024-02-07T13:34:38.349981+0000","last_undegraded":"2024-02-07T13:34:38.349981+0000","last_fullsized":"2024-02-07T13:34:38.349981+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:37:26.784644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353781+0000","last_change":"2024-02-07T13:34:17.448344+0000","last_active":"2024-02-07T13:34:38.353781+0000","last_peered":"2024-02-07T13:34:38.353781+0000","last_clean":"2024-02-07T13:34:38.353781+0000","last_became_active":"2024-02-07T13:34:17.447797+0000","last_became_peered":"2024-02-07T13:34:17.447797+0000","last_unstale":"2024-02-07T13:34:38.353781+0000","last_undegraded":"2024-02-07T13:34:38.353781+0000","last_fullsized":"2024-02-07T13:34:38.353781+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:09:14.154285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352598+0000","last_change":"2024-02-07T13:34:15.434361+0000","last_active":"2024-02-07T13:34:38.352598+0000","last_peered":"2024-02-07T13:34:38.352598+0000","last_clean":"2024-02-07T13:34:38.352598+0000","last_became_active":"2024-02-07T13:34:15.433807+0000","last_became_peered":"2024-02-07T13:34:15.433807+0000","last_unstale":"2024-02-07T13:34:38.352598+0000","last_undegraded":"2024-02-07T13:34:38.352598+0000","last_fullsized":"2024-02-07T13:34:38.352598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:03:20.395174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354966+0000","last_change":"2024-02-07T13:34:11.420629+0000","last_active":"2024-02-07T13:34:38.354966+0000","last_peered":"2024-02-07T13:34:38.354966+0000","last_clean":"2024-02-07T13:34:38.354966+0000","last_became_active":"2024-02-07T13:34:11.420329+0000","last_became_peered":"2024-02-07T13:34:11.420329+0000","last_unstale":"2024-02-07T13:34:38.354966+0000","last_undegraded":"2024-02-07T13:34:38.354966+0000","last_fullsized":"2024-02-07T13:34:38.354966+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:29:19.756265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.349998+0000","last_change":"2024-02-07T13:34:13.430211+0000","last_active":"2024-02-07T13:34:38.349998+0000","last_peered":"2024-02-07T13:34:38.349998+0000","last_clean":"2024-02-07T13:34:38.349998+0000","last_became_active":"2024-02-07T13:34:13.428474+0000","last_became_peered":"2024-02-07T13:34:13.428474+0000","last_unstale":"2024-02-07T13:34:38.349998+0000","last_undegraded":"2024-02-07T13:34:38.349998+0000","last_fullsized":"2024-02-07T13:34:38.349998+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:40:27.223364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352851+0000","last_change":"2024-02-07T13:34:17.446797+0000","last_active":"2024-02-07T13:34:38.352851+0000","last_peered":"2024-02-07T13:34:38.352851+0000","last_clean":"2024-02-07T13:34:38.352851+0000","last_became_active":"2024-02-07T13:34:17.446445+0000","last_became_peered":"2024-02-07T13:34:17.446445+0000","last_unstale":"2024-02-07T13:34:38.352851+0000","last_undegraded":"2024-02-07T13:34:38.352851+0000","last_fullsized":"2024-02-07T13:34:38.352851+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:36:24.011394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.349893+0000","last_change":"2024-02-07T13:34:15.434239+0000","last_active":"2024-02-07T13:34:38.349893+0000","last_peered":"2024-02-07T13:34:38.349893+0000","last_clean":"2024-02-07T13:34:38.349893+0000","last_became_active":"2024-02-07T13:34:15.433878+0000","last_became_peered":"2024-02-07T13:34:15.433878+0000","last_unstale":"2024-02-07T13:34:38.349893+0000","last_undegraded":"2024-02-07T13:34:38.349893+0000","last_fullsized":"2024-02-07T13:34:38.349893+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:25:57.461991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354964+0000","last_change":"2024-02-07T13:34:11.424045+0000","last_active":"2024-02-07T13:34:38.354964+0000","last_peered":"2024-02-07T13:34:38.354964+0000","last_clean":"2024-02-07T13:34:38.354964+0000","last_became_active":"2024-02-07T13:34:11.423733+0000","last_became_peered":"2024-02-07T13:34:11.423733+0000","last_unstale":"2024-02-07T13:34:38.354964+0000","last_undegraded":"2024-02-07T13:34:38.354964+0000","last_fullsized":"2024-02-07T13:34:38.354964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:53:57.469051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353767+0000","last_change":"2024-02-07T13:34:13.429172+0000","last_active":"2024-02-07T13:34:38.353767+0000","last_peered":"2024-02-07T13:34:38.353767+0000","last_clean":"2024-02-07T13:34:38.353767+0000","last_became_active":"2024-02-07T13:34:13.428849+0000","last_became_peered":"2024-02-07T13:34:13.428849+0000","last_unstale":"2024-02-07T13:34:38.353767+0000","last_undegraded":"2024-02-07T13:34:38.353767+0000","last_fullsized":"2024-02-07T13:34:38.353767+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:05:08.010111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352519+0000","last_change":"2024-02-07T13:34:17.447233+0000","last_active":"2024-02-07T13:34:38.352519+0000","last_peered":"2024-02-07T13:34:38.352519+0000","last_clean":"2024-02-07T13:34:38.352519+0000","last_became_active":"2024-02-07T13:34:17.446406+0000","last_became_peered":"2024-02-07T13:34:17.446406+0000","last_unstale":"2024-02-07T13:34:38.352519+0000","last_undegraded":"2024-02-07T13:34:38.352519+0000","last_fullsized":"2024-02-07T13:34:38.352519+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:00:00.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350441+0000","last_change":"2024-02-07T13:34:15.436466+0000","last_active":"2024-02-07T13:34:38.350441+0000","last_peered":"2024-02-07T13:34:38.350441+0000","last_clean":"2024-02-07T13:34:38.350441+0000","last_became_active":"2024-02-07T13:34:15.436081+0000","last_became_peered":"2024-02-07T13:34:15.436081+0000","last_unstale":"2024-02-07T13:34:38.350441+0000","last_undegraded":"2024-02-07T13:34:38.350441+0000","last_fullsized":"2024-02-07T13:34:38.350441+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:57:44.431175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353601+0000","last_change":"2024-02-07T13:34:11.424461+0000","last_active":"2024-02-07T13:34:38.353601+0000","last_peered":"2024-02-07T13:34:38.353601+0000","last_clean":"2024-02-07T13:34:38.353601+0000","last_became_active":"2024-02-07T13:34:11.424092+0000","last_became_peered":"2024-02-07T13:34:11.424092+0000","last_unstale":"2024-02-07T13:34:38.353601+0000","last_undegraded":"2024-02-07T13:34:38.353601+0000","last_fullsized":"2024-02-07T13:34:38.353601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:42:59.389803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354771+0000","last_change":"2024-02-07T13:34:13.430371+0000","last_active":"2024-02-07T13:34:38.354771+0000","last_peered":"2024-02-07T13:34:38.354771+0000","last_clean":"2024-02-07T13:34:38.354771+0000","last_became_active":"2024-02-07T13:34:13.430062+0000","last_became_peered":"2024-02-07T13:34:13.430062+0000","last_unstale":"2024-02-07T13:34:38.354771+0000","last_undegraded":"2024-02-07T13:34:38.354771+0000","last_fullsized":"2024-02-07T13:34:38.354771+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:22:43.486630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355366+0000","last_change":"2024-02-07T13:34:17.448727+0000","last_active":"2024-02-07T13:34:38.355366+0000","last_peered":"2024-02-07T13:34:38.355366+0000","last_clean":"2024-02-07T13:34:38.355366+0000","last_became_active":"2024-02-07T13:34:17.448423+0000","last_became_peered":"2024-02-07T13:34:17.448423+0000","last_unstale":"2024-02-07T13:34:38.355366+0000","last_undegraded":"2024-02-07T13:34:38.355366+0000","last_fullsized":"2024-02-07T13:34:38.355366+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:01:44.308094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353176+0000","last_change":"2024-02-07T13:34:15.437267+0000","last_active":"2024-02-07T13:34:38.353176+0000","last_peered":"2024-02-07T13:34:38.353176+0000","last_clean":"2024-02-07T13:34:38.353176+0000","last_became_active":"2024-02-07T13:34:15.436792+0000","last_became_peered":"2024-02-07T13:34:15.436792+0000","last_unstale":"2024-02-07T13:34:38.353176+0000","last_undegraded":"2024-02-07T13:34:38.353176+0000","last_fullsized":"2024-02-07T13:34:38.353176+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:39:52.476866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356947+0000","last_change":"2024-02-07T13:34:11.420689+0000","last_active":"2024-02-07T13:34:38.356947+0000","last_peered":"2024-02-07T13:34:38.356947+0000","last_clean":"2024-02-07T13:34:38.356947+0000","last_became_active":"2024-02-07T13:34:11.419927+0000","last_became_peered":"2024-02-07T13:34:11.419927+0000","last_unstale":"2024-02-07T13:34:38.356947+0000","last_undegraded":"2024-02-07T13:34:38.356947+0000","last_fullsized":"2024-02-07T13:34:38.356947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:09:19.994316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355167+0000","last_change":"2024-02-07T13:34:13.427809+0000","last_active":"2024-02-07T13:34:38.355167+0000","last_peered":"2024-02-07T13:34:38.355167+0000","last_clean":"2024-02-07T13:34:38.355167+0000","last_became_active":"2024-02-07T13:34:13.427487+0000","last_became_peered":"2024-02-07T13:34:13.427487+0000","last_unstale":"2024-02-07T13:34:38.355167+0000","last_undegraded":"2024-02-07T13:34:38.355167+0000","last_fullsized":"2024-02-07T13:34:38.355167+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:55:06.630474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355372+0000","last_change":"2024-02-07T13:34:17.448846+0000","last_active":"2024-02-07T13:34:38.355372+0000","last_peered":"2024-02-07T13:34:38.355372+0000","last_clean":"2024-02-07T13:34:38.355372+0000","last_became_active":"2024-02-07T13:34:17.448603+0000","last_became_peered":"2024-02-07T13:34:17.448603+0000","last_unstale":"2024-02-07T13:34:38.355372+0000","last_undegraded":"2024-02-07T13:34:38.355372+0000","last_fullsized":"2024-02-07T13:34:38.355372+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:45:58.102122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350056+0000","last_change":"2024-02-07T13:34:15.434590+0000","last_active":"2024-02-07T13:34:38.350056+0000","last_peered":"2024-02-07T13:34:38.350056+0000","last_clean":"2024-02-07T13:34:38.350056+0000","last_became_active":"2024-02-07T13:34:15.434268+0000","last_became_peered":"2024-02-07T13:34:15.434268+0000","last_unstale":"2024-02-07T13:34:38.350056+0000","last_undegraded":"2024-02-07T13:34:38.350056+0000","last_fullsized":"2024-02-07T13:34:38.350056+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:24:00.707025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355077+0000","last_change":"2024-02-07T13:34:11.424011+0000","last_active":"2024-02-07T13:34:38.355077+0000","last_peered":"2024-02-07T13:34:38.355077+0000","last_clean":"2024-02-07T13:34:38.355077+0000","last_became_active":"2024-02-07T13:34:11.423566+0000","last_became_peered":"2024-02-07T13:34:11.423566+0000","last_unstale":"2024-02-07T13:34:38.355077+0000","last_undegraded":"2024-02-07T13:34:38.355077+0000","last_fullsized":"2024-02-07T13:34:38.355077+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:32:37.801622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356743+0000","last_change":"2024-02-07T13:34:13.435131+0000","last_active":"2024-02-07T13:34:38.356743+0000","last_peered":"2024-02-07T13:34:38.356743+0000","last_clean":"2024-02-07T13:34:38.356743+0000","last_became_active":"2024-02-07T13:34:13.434670+0000","last_became_peered":"2024-02-07T13:34:13.434670+0000","last_unstale":"2024-02-07T13:34:38.356743+0000","last_undegraded":"2024-02-07T13:34:38.356743+0000","last_fullsized":"2024-02-07T13:34:38.356743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:25:26.379511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352880+0000","last_change":"2024-02-07T13:34:17.446281+0000","last_active":"2024-02-07T13:34:38.352880+0000","last_peered":"2024-02-07T13:34:38.352880+0000","last_clean":"2024-02-07T13:34:38.352880+0000","last_became_active":"2024-02-07T13:34:17.445870+0000","last_became_peered":"2024-02-07T13:34:17.445870+0000","last_unstale":"2024-02-07T13:34:38.352880+0000","last_undegraded":"2024-02-07T13:34:38.352880+0000","last_fullsized":"2024-02-07T13:34:38.352880+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:35:26.733830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469606+0000","last_change":"2024-02-07T13:34:15.432427+0000","last_active":"2024-02-07T13:34:46.469606+0000","last_peered":"2024-02-07T13:34:46.469606+0000","last_clean":"2024-02-07T13:34:46.469606+0000","last_became_active":"2024-02-07T13:34:15.432134+0000","last_became_peered":"2024-02-07T13:34:15.432134+0000","last_unstale":"2024-02-07T13:34:46.469606+0000","last_undegraded":"2024-02-07T13:34:46.469606+0000","last_fullsized":"2024-02-07T13:34:46.469606+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:15:50.805974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355902+0000","last_change":"2024-02-07T13:34:11.420900+0000","last_active":"2024-02-07T13:34:38.355902+0000","last_peered":"2024-02-07T13:34:38.355902+0000","last_clean":"2024-02-07T13:34:38.355902+0000","last_became_active":"2024-02-07T13:34:11.420383+0000","last_became_peered":"2024-02-07T13:34:11.420383+0000","last_unstale":"2024-02-07T13:34:38.355902+0000","last_undegraded":"2024-02-07T13:34:38.355902+0000","last_fullsized":"2024-02-07T13:34:38.355902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:58:47.580443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350442+0000","last_change":"2024-02-07T13:34:13.427434+0000","last_active":"2024-02-07T13:34:38.350442+0000","last_peered":"2024-02-07T13:34:38.350442+0000","last_clean":"2024-02-07T13:34:38.350442+0000","last_became_active":"2024-02-07T13:34:13.427074+0000","last_became_peered":"2024-02-07T13:34:13.427074+0000","last_unstale":"2024-02-07T13:34:38.350442+0000","last_undegraded":"2024-02-07T13:34:38.350442+0000","last_fullsized":"2024-02-07T13:34:38.350442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:04:47.455644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352933+0000","last_change":"2024-02-07T13:34:17.447043+0000","last_active":"2024-02-07T13:34:38.352933+0000","last_peered":"2024-02-07T13:34:38.352933+0000","last_clean":"2024-02-07T13:34:38.352933+0000","last_became_active":"2024-02-07T13:34:17.446227+0000","last_became_peered":"2024-02-07T13:34:17.446227+0000","last_unstale":"2024-02-07T13:34:38.352933+0000","last_undegraded":"2024-02-07T13:34:38.352933+0000","last_fullsized":"2024-02-07T13:34:38.352933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:29:56.967993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469630+0000","last_change":"2024-02-07T13:34:15.433876+0000","last_active":"2024-02-07T13:34:46.469630+0000","last_peered":"2024-02-07T13:34:46.469630+0000","last_clean":"2024-02-07T13:34:46.469630+0000","last_became_active":"2024-02-07T13:34:15.432776+0000","last_became_peered":"2024-02-07T13:34:15.432776+0000","last_unstale":"2024-02-07T13:34:46.469630+0000","last_undegraded":"2024-02-07T13:34:46.469630+0000","last_fullsized":"2024-02-07T13:34:46.469630+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:39:46.118469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355610+0000","last_change":"2024-02-07T13:34:11.419418+0000","last_active":"2024-02-07T13:34:38.355610+0000","last_peered":"2024-02-07T13:34:38.355610+0000","last_clean":"2024-02-07T13:34:38.355610+0000","last_became_active":"2024-02-07T13:34:11.419046+0000","last_became_peered":"2024-02-07T13:34:11.419046+0000","last_unstale":"2024-02-07T13:34:38.355610+0000","last_undegraded":"2024-02-07T13:34:38.355610+0000","last_fullsized":"2024-02-07T13:34:38.355610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:28:35.600401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353389+0000","last_change":"2024-02-07T13:34:13.428022+0000","last_active":"2024-02-07T13:34:38.353389+0000","last_peered":"2024-02-07T13:34:38.353389+0000","last_clean":"2024-02-07T13:34:38.353389+0000","last_became_active":"2024-02-07T13:34:13.427623+0000","last_became_peered":"2024-02-07T13:34:13.427623+0000","last_unstale":"2024-02-07T13:34:38.353389+0000","last_undegraded":"2024-02-07T13:34:38.353389+0000","last_fullsized":"2024-02-07T13:34:38.353389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:43:06.553569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352934+0000","last_change":"2024-02-07T13:34:17.447973+0000","last_active":"2024-02-07T13:34:38.352934+0000","last_peered":"2024-02-07T13:34:38.352934+0000","last_clean":"2024-02-07T13:34:38.352934+0000","last_became_active":"2024-02-07T13:34:17.446117+0000","last_became_peered":"2024-02-07T13:34:17.446117+0000","last_unstale":"2024-02-07T13:34:38.352934+0000","last_undegraded":"2024-02-07T13:34:38.352934+0000","last_fullsized":"2024-02-07T13:34:38.352934+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:15:28.589183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353243+0000","last_change":"2024-02-07T13:34:15.435544+0000","last_active":"2024-02-07T13:34:38.353243+0000","last_peered":"2024-02-07T13:34:38.353243+0000","last_clean":"2024-02-07T13:34:38.353243+0000","last_became_active":"2024-02-07T13:34:15.434899+0000","last_became_peered":"2024-02-07T13:34:15.434899+0000","last_unstale":"2024-02-07T13:34:38.353243+0000","last_undegraded":"2024-02-07T13:34:38.353243+0000","last_fullsized":"2024-02-07T13:34:38.353243+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:33:55.440321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352417+0000","last_change":"2024-02-07T13:34:11.419779+0000","last_active":"2024-02-07T13:34:38.352417+0000","last_peered":"2024-02-07T13:34:38.352417+0000","last_clean":"2024-02-07T13:34:38.352417+0000","last_became_active":"2024-02-07T13:34:11.419445+0000","last_became_peered":"2024-02-07T13:34:11.419445+0000","last_unstale":"2024-02-07T13:34:38.352417+0000","last_undegraded":"2024-02-07T13:34:38.352417+0000","last_fullsized":"2024-02-07T13:34:38.352417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:32:12.627238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352911+0000","last_change":"2024-02-07T13:34:13.429901+0000","last_active":"2024-02-07T13:34:38.352911+0000","last_peered":"2024-02-07T13:34:38.352911+0000","last_clean":"2024-02-07T13:34:38.352911+0000","last_became_active":"2024-02-07T13:34:13.428705+0000","last_became_peered":"2024-02-07T13:34:13.428705+0000","last_unstale":"2024-02-07T13:34:38.352911+0000","last_undegraded":"2024-02-07T13:34:38.352911+0000","last_fullsized":"2024-02-07T13:34:38.352911+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:17:00.563014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352981+0000","last_change":"2024-02-07T13:34:17.446627+0000","last_active":"2024-02-07T13:34:38.352981+0000","last_peered":"2024-02-07T13:34:38.352981+0000","last_clean":"2024-02-07T13:34:38.352981+0000","last_became_active":"2024-02-07T13:34:17.446175+0000","last_became_peered":"2024-02-07T13:34:17.446175+0000","last_unstale":"2024-02-07T13:34:38.352981+0000","last_undegraded":"2024-02-07T13:34:38.352981+0000","last_fullsized":"2024-02-07T13:34:38.352981+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:54:10.152159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352678+0000","last_change":"2024-02-07T13:34:15.435390+0000","last_active":"2024-02-07T13:34:38.352678+0000","last_peered":"2024-02-07T13:34:38.352678+0000","last_clean":"2024-02-07T13:34:38.352678+0000","last_became_active":"2024-02-07T13:34:15.434805+0000","last_became_peered":"2024-02-07T13:34:15.434805+0000","last_unstale":"2024-02-07T13:34:38.352678+0000","last_undegraded":"2024-02-07T13:34:38.352678+0000","last_fullsized":"2024-02-07T13:34:38.352678+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:56:30.098575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355137+0000","last_change":"2024-02-07T13:34:11.425143+0000","last_active":"2024-02-07T13:34:38.355137+0000","last_peered":"2024-02-07T13:34:38.355137+0000","last_clean":"2024-02-07T13:34:38.355137+0000","last_became_active":"2024-02-07T13:34:11.424648+0000","last_became_peered":"2024-02-07T13:34:11.424648+0000","last_unstale":"2024-02-07T13:34:38.355137+0000","last_undegraded":"2024-02-07T13:34:38.355137+0000","last_fullsized":"2024-02-07T13:34:38.355137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:54:57.242656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355112+0000","last_change":"2024-02-07T13:34:13.432344+0000","last_active":"2024-02-07T13:34:38.355112+0000","last_peered":"2024-02-07T13:34:38.355112+0000","last_clean":"2024-02-07T13:34:38.355112+0000","last_became_active":"2024-02-07T13:34:13.431703+0000","last_became_peered":"2024-02-07T13:34:13.431703+0000","last_unstale":"2024-02-07T13:34:38.355112+0000","last_undegraded":"2024-02-07T13:34:38.355112+0000","last_fullsized":"2024-02-07T13:34:38.355112+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:57:56.365302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355722+0000","last_change":"2024-02-07T13:34:17.448487+0000","last_active":"2024-02-07T13:34:38.355722+0000","last_peered":"2024-02-07T13:34:38.355722+0000","last_clean":"2024-02-07T13:34:38.355722+0000","last_became_active":"2024-02-07T13:34:17.448238+0000","last_became_peered":"2024-02-07T13:34:17.448238+0000","last_unstale":"2024-02-07T13:34:38.355722+0000","last_undegraded":"2024-02-07T13:34:38.355722+0000","last_fullsized":"2024-02-07T13:34:38.355722+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:29:30.060230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357668+0000","last_change":"2024-02-07T13:34:11.419648+0000","last_active":"2024-02-07T13:34:38.357668+0000","last_peered":"2024-02-07T13:34:38.357668+0000","last_clean":"2024-02-07T13:34:38.357668+0000","last_became_active":"2024-02-07T13:34:11.419041+0000","last_became_peered":"2024-02-07T13:34:11.419041+0000","last_unstale":"2024-02-07T13:34:38.357668+0000","last_undegraded":"2024-02-07T13:34:38.357668+0000","last_fullsized":"2024-02-07T13:34:38.357668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:36:57.130050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357413+0000","last_change":"2024-02-07T13:34:13.435296+0000","last_active":"2024-02-07T13:34:38.357413+0000","last_peered":"2024-02-07T13:34:38.357413+0000","last_clean":"2024-02-07T13:34:38.357413+0000","last_became_active":"2024-02-07T13:34:13.435017+0000","last_became_peered":"2024-02-07T13:34:13.435017+0000","last_unstale":"2024-02-07T13:34:38.357413+0000","last_undegraded":"2024-02-07T13:34:38.357413+0000","last_fullsized":"2024-02-07T13:34:38.357413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:34:00.419145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469636+0000","last_change":"2024-02-07T13:34:15.432735+0000","last_active":"2024-02-07T13:34:46.469636+0000","last_peered":"2024-02-07T13:34:46.469636+0000","last_clean":"2024-02-07T13:34:46.469636+0000","last_became_active":"2024-02-07T13:34:15.432451+0000","last_became_peered":"2024-02-07T13:34:15.432451+0000","last_unstale":"2024-02-07T13:34:46.469636+0000","last_undegraded":"2024-02-07T13:34:46.469636+0000","last_fullsized":"2024-02-07T13:34:46.469636+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:26:19.853005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350352+0000","last_change":"2024-02-07T13:34:17.444006+0000","last_active":"2024-02-07T13:34:38.350352+0000","last_peered":"2024-02-07T13:34:38.350352+0000","last_clean":"2024-02-07T13:34:38.350352+0000","last_became_active":"2024-02-07T13:34:17.443701+0000","last_became_peered":"2024-02-07T13:34:17.443701+0000","last_unstale":"2024-02-07T13:34:38.350352+0000","last_undegraded":"2024-02-07T13:34:38.350352+0000","last_fullsized":"2024-02-07T13:34:38.350352+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:05:09.757974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353127+0000","last_change":"2024-02-07T13:34:11.420704+0000","last_active":"2024-02-07T13:34:38.353127+0000","last_peered":"2024-02-07T13:34:38.353127+0000","last_clean":"2024-02-07T13:34:38.353127+0000","last_became_active":"2024-02-07T13:34:11.420266+0000","last_became_peered":"2024-02-07T13:34:11.420266+0000","last_unstale":"2024-02-07T13:34:38.353127+0000","last_undegraded":"2024-02-07T13:34:38.353127+0000","last_fullsized":"2024-02-07T13:34:38.353127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:04:01.787605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469441+0000","last_change":"2024-02-07T13:34:13.429780+0000","last_active":"2024-02-07T13:34:46.469441+0000","last_peered":"2024-02-07T13:34:46.469441+0000","last_clean":"2024-02-07T13:34:46.469441+0000","last_became_active":"2024-02-07T13:34:13.428647+0000","last_became_peered":"2024-02-07T13:34:13.428647+0000","last_unstale":"2024-02-07T13:34:46.469441+0000","last_undegraded":"2024-02-07T13:34:46.469441+0000","last_fullsized":"2024-02-07T13:34:46.469441+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:04:53.190980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355460+0000","last_change":"2024-02-07T13:34:15.430738+0000","last_active":"2024-02-07T13:34:38.355460+0000","last_peered":"2024-02-07T13:34:38.355460+0000","last_clean":"2024-02-07T13:34:38.355460+0000","last_became_active":"2024-02-07T13:34:15.430476+0000","last_became_peered":"2024-02-07T13:34:15.430476+0000","last_unstale":"2024-02-07T13:34:38.355460+0000","last_undegraded":"2024-02-07T13:34:38.355460+0000","last_fullsized":"2024-02-07T13:34:38.355460+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:34:26.724082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350821+0000","last_change":"2024-02-07T13:34:17.446368+0000","last_active":"2024-02-07T13:34:38.350821+0000","last_peered":"2024-02-07T13:34:38.350821+0000","last_clean":"2024-02-07T13:34:38.350821+0000","last_became_active":"2024-02-07T13:34:17.446027+0000","last_became_peered":"2024-02-07T13:34:17.446027+0000","last_unstale":"2024-02-07T13:34:38.350821+0000","last_undegraded":"2024-02-07T13:34:38.350821+0000","last_fullsized":"2024-02-07T13:34:38.350821+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:02:56.944115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352902+0000","last_change":"2024-02-07T13:34:11.423264+0000","last_active":"2024-02-07T13:34:38.352902+0000","last_peered":"2024-02-07T13:34:38.352902+0000","last_clean":"2024-02-07T13:34:38.352902+0000","last_became_active":"2024-02-07T13:34:11.422917+0000","last_became_peered":"2024-02-07T13:34:11.422917+0000","last_unstale":"2024-02-07T13:34:38.352902+0000","last_undegraded":"2024-02-07T13:34:38.352902+0000","last_fullsized":"2024-02-07T13:34:38.352902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:35:25.720896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354734+0000","last_change":"2024-02-07T13:34:13.428004+0000","last_active":"2024-02-07T13:34:38.354734+0000","last_peered":"2024-02-07T13:34:38.354734+0000","last_clean":"2024-02-07T13:34:38.354734+0000","last_became_active":"2024-02-07T13:34:13.427660+0000","last_became_peered":"2024-02-07T13:34:13.427660+0000","last_unstale":"2024-02-07T13:34:38.354734+0000","last_undegraded":"2024-02-07T13:34:38.354734+0000","last_fullsized":"2024-02-07T13:34:38.354734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:14:38.543175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352973+0000","last_change":"2024-02-07T13:34:15.434731+0000","last_active":"2024-02-07T13:34:38.352973+0000","last_peered":"2024-02-07T13:34:38.352973+0000","last_clean":"2024-02-07T13:34:38.352973+0000","last_became_active":"2024-02-07T13:34:15.434387+0000","last_became_peered":"2024-02-07T13:34:15.434387+0000","last_unstale":"2024-02-07T13:34:38.352973+0000","last_undegraded":"2024-02-07T13:34:38.352973+0000","last_fullsized":"2024-02-07T13:34:38.352973+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:14:27.602695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353479+0000","last_change":"2024-02-07T13:34:17.449515+0000","last_active":"2024-02-07T13:34:38.353479+0000","last_peered":"2024-02-07T13:34:38.353479+0000","last_clean":"2024-02-07T13:34:38.353479+0000","last_became_active":"2024-02-07T13:34:17.449250+0000","last_became_peered":"2024-02-07T13:34:17.449250+0000","last_unstale":"2024-02-07T13:34:38.353479+0000","last_undegraded":"2024-02-07T13:34:38.353479+0000","last_fullsized":"2024-02-07T13:34:38.353479+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:49:58.354627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354807+0000","last_change":"2024-02-07T13:34:13.435045+0000","last_active":"2024-02-07T13:34:38.354807+0000","last_peered":"2024-02-07T13:34:38.354807+0000","last_clean":"2024-02-07T13:34:38.354807+0000","last_became_active":"2024-02-07T13:34:13.434538+0000","last_became_peered":"2024-02-07T13:34:13.434538+0000","last_unstale":"2024-02-07T13:34:38.354807+0000","last_undegraded":"2024-02-07T13:34:38.354807+0000","last_fullsized":"2024-02-07T13:34:38.354807+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:46:13.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350085+0000","last_change":"2024-02-07T13:34:11.422066+0000","last_active":"2024-02-07T13:34:38.350085+0000","last_peered":"2024-02-07T13:34:38.350085+0000","last_clean":"2024-02-07T13:34:38.350085+0000","last_became_active":"2024-02-07T13:34:11.420383+0000","last_became_peered":"2024-02-07T13:34:11.420383+0000","last_unstale":"2024-02-07T13:34:38.350085+0000","last_undegraded":"2024-02-07T13:34:38.350085+0000","last_fullsized":"2024-02-07T13:34:38.350085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:54:06.859637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354764+0000","last_change":"2024-02-07T13:34:15.436526+0000","last_active":"2024-02-07T13:34:38.354764+0000","last_peered":"2024-02-07T13:34:38.354764+0000","last_clean":"2024-02-07T13:34:38.354764+0000","last_became_active":"2024-02-07T13:34:15.436146+0000","last_became_peered":"2024-02-07T13:34:15.436146+0000","last_unstale":"2024-02-07T13:34:38.354764+0000","last_undegraded":"2024-02-07T13:34:38.354764+0000","last_fullsized":"2024-02-07T13:34:38.354764+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:44:33.250516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355136+0000","last_change":"2024-02-07T13:34:17.448268+0000","last_active":"2024-02-07T13:34:38.355136+0000","last_peered":"2024-02-07T13:34:38.355136+0000","last_clean":"2024-02-07T13:34:38.355136+0000","last_became_active":"2024-02-07T13:34:17.447973+0000","last_became_peered":"2024-02-07T13:34:17.447973+0000","last_unstale":"2024-02-07T13:34:38.355136+0000","last_undegraded":"2024-02-07T13:34:38.355136+0000","last_fullsized":"2024-02-07T13:34:38.355136+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:48:49.028545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350213+0000","last_change":"2024-02-07T13:34:13.429105+0000","last_active":"2024-02-07T13:34:38.350213+0000","last_peered":"2024-02-07T13:34:38.350213+0000","last_clean":"2024-02-07T13:34:38.350213+0000","last_became_active":"2024-02-07T13:34:13.428463+0000","last_became_peered":"2024-02-07T13:34:13.428463+0000","last_unstale":"2024-02-07T13:34:38.350213+0000","last_undegraded":"2024-02-07T13:34:38.350213+0000","last_fullsized":"2024-02-07T13:34:38.350213+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:28:06.020866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353581+0000","last_change":"2024-02-07T13:34:11.419837+0000","last_active":"2024-02-07T13:34:38.353581+0000","last_peered":"2024-02-07T13:34:38.353581+0000","last_clean":"2024-02-07T13:34:38.353581+0000","last_became_active":"2024-02-07T13:34:11.419461+0000","last_became_peered":"2024-02-07T13:34:11.419461+0000","last_unstale":"2024-02-07T13:34:38.353581+0000","last_undegraded":"2024-02-07T13:34:38.353581+0000","last_fullsized":"2024-02-07T13:34:38.353581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:53:54.714546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:40.453295+0000","last_change":"2024-02-07T13:34:00.029303+0000","last_active":"2024-02-07T13:34:40.453295+0000","last_peered":"2024-02-07T13:34:40.453295+0000","last_clean":"2024-02-07T13:34:40.453295+0000","last_became_active":"2024-02-07T13:34:00.028931+0000","last_became_peered":"2024-02-07T13:34:00.028931+0000","last_unstale":"2024-02-07T13:34:40.453295+0000","last_undegraded":"2024-02-07T13:34:40.453295+0000","last_fullsized":"2024-02-07T13:34:40.453295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:33:59.019138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:33:59.019138+0000","last_clean_scrub_stamp":"2024-02-07T13:33:59.019138+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:00:38.266935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353264+0000","last_change":"2024-02-07T13:34:15.437382+0000","last_active":"2024-02-07T13:34:38.353264+0000","last_peered":"2024-02-07T13:34:38.353264+0000","last_clean":"2024-02-07T13:34:38.353264+0000","last_became_active":"2024-02-07T13:34:15.436988+0000","last_became_peered":"2024-02-07T13:34:15.436988+0000","last_unstale":"2024-02-07T13:34:38.353264+0000","last_undegraded":"2024-02-07T13:34:38.353264+0000","last_fullsized":"2024-02-07T13:34:38.353264+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:10:23.455203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353783+0000","last_change":"2024-02-07T13:34:17.448578+0000","last_active":"2024-02-07T13:34:38.353783+0000","last_peered":"2024-02-07T13:34:38.353783+0000","last_clean":"2024-02-07T13:34:38.353783+0000","last_became_active":"2024-02-07T13:34:17.448120+0000","last_became_peered":"2024-02-07T13:34:17.448120+0000","last_unstale":"2024-02-07T13:34:38.353783+0000","last_undegraded":"2024-02-07T13:34:38.353783+0000","last_fullsized":"2024-02-07T13:34:38.353783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:35:35.025820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350702+0000","last_change":"2024-02-07T13:34:13.427655+0000","last_active":"2024-02-07T13:34:38.350702+0000","last_peered":"2024-02-07T13:34:38.350702+0000","last_clean":"2024-02-07T13:34:38.350702+0000","last_became_active":"2024-02-07T13:34:13.427289+0000","last_became_peered":"2024-02-07T13:34:13.427289+0000","last_unstale":"2024-02-07T13:34:38.350702+0000","last_undegraded":"2024-02-07T13:34:38.350702+0000","last_fullsized":"2024-02-07T13:34:38.350702+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:22:15.562439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352248+0000","last_change":"2024-02-07T13:34:11.425397+0000","last_active":"2024-02-07T13:34:38.352248+0000","last_peered":"2024-02-07T13:34:38.352248+0000","last_clean":"2024-02-07T13:34:38.352248+0000","last_became_active":"2024-02-07T13:34:11.424195+0000","last_became_peered":"2024-02-07T13:34:11.424195+0000","last_unstale":"2024-02-07T13:34:38.352248+0000","last_undegraded":"2024-02-07T13:34:38.352248+0000","last_fullsized":"2024-02-07T13:34:38.352248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:42:06.609380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357041+0000","last_change":"2024-02-07T13:34:15.431879+0000","last_active":"2024-02-07T13:34:38.357041+0000","last_peered":"2024-02-07T13:34:38.357041+0000","last_clean":"2024-02-07T13:34:38.357041+0000","last_became_active":"2024-02-07T13:34:15.431472+0000","last_became_peered":"2024-02-07T13:34:15.431472+0000","last_unstale":"2024-02-07T13:34:38.357041+0000","last_undegraded":"2024-02-07T13:34:38.357041+0000","last_fullsized":"2024-02-07T13:34:38.357041+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:32:45.940185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352759+0000","last_change":"2024-02-07T13:34:17.445806+0000","last_active":"2024-02-07T13:34:38.352759+0000","last_peered":"2024-02-07T13:34:38.352759+0000","last_clean":"2024-02-07T13:34:38.352759+0000","last_became_active":"2024-02-07T13:34:17.444983+0000","last_became_peered":"2024-02-07T13:34:17.444983+0000","last_unstale":"2024-02-07T13:34:38.352759+0000","last_undegraded":"2024-02-07T13:34:38.352759+0000","last_fullsized":"2024-02-07T13:34:38.352759+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:10:29.068022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356882+0000","last_change":"2024-02-07T13:34:11.420574+0000","last_active":"2024-02-07T13:34:38.356882+0000","last_peered":"2024-02-07T13:34:38.356882+0000","last_clean":"2024-02-07T13:34:38.356882+0000","last_became_active":"2024-02-07T13:34:11.419823+0000","last_became_peered":"2024-02-07T13:34:11.419823+0000","last_unstale":"2024-02-07T13:34:38.356882+0000","last_undegraded":"2024-02-07T13:34:38.356882+0000","last_fullsized":"2024-02-07T13:34:38.356882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:17:21.983253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":80,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469286+0000","last_change":"2024-02-07T13:34:13.430034+0000","last_active":"2024-02-07T13:34:46.469286+0000","last_peered":"2024-02-07T13:34:46.469286+0000","last_clean":"2024-02-07T13:34:46.469286+0000","last_became_active":"2024-02-07T13:34:13.429299+0000","last_became_peered":"2024-02-07T13:34:13.429299+0000","last_unstale":"2024-02-07T13:34:46.469286+0000","last_undegraded":"2024-02-07T13:34:46.469286+0000","last_fullsized":"2024-02-07T13:34:46.469286+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:14:03.010029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469590+0000","last_change":"2024-02-07T13:34:15.431210+0000","last_active":"2024-02-07T13:34:46.469590+0000","last_peered":"2024-02-07T13:34:46.469590+0000","last_clean":"2024-02-07T13:34:46.469590+0000","last_became_active":"2024-02-07T13:34:15.430915+0000","last_became_peered":"2024-02-07T13:34:15.430915+0000","last_unstale":"2024-02-07T13:34:46.469590+0000","last_undegraded":"2024-02-07T13:34:46.469590+0000","last_fullsized":"2024-02-07T13:34:46.469590+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:08:19.151911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352151+0000","last_change":"2024-02-07T13:34:17.445783+0000","last_active":"2024-02-07T13:34:38.352151+0000","last_peered":"2024-02-07T13:34:38.352151+0000","last_clean":"2024-02-07T13:34:38.352151+0000","last_became_active":"2024-02-07T13:34:17.445059+0000","last_became_peered":"2024-02-07T13:34:17.445059+0000","last_unstale":"2024-02-07T13:34:38.352151+0000","last_undegraded":"2024-02-07T13:34:38.352151+0000","last_fullsized":"2024-02-07T13:34:38.352151+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:53:23.596324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353242+0000","last_change":"2024-02-07T13:34:11.420856+0000","last_active":"2024-02-07T13:34:38.353242+0000","last_peered":"2024-02-07T13:34:38.353242+0000","last_clean":"2024-02-07T13:34:38.353242+0000","last_became_active":"2024-02-07T13:34:11.419976+0000","last_became_peered":"2024-02-07T13:34:11.419976+0000","last_unstale":"2024-02-07T13:34:38.353242+0000","last_undegraded":"2024-02-07T13:34:38.353242+0000","last_fullsized":"2024-02-07T13:34:38.353242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:50:24.307168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354902+0000","last_change":"2024-02-07T13:34:13.429770+0000","last_active":"2024-02-07T13:34:38.354902+0000","last_peered":"2024-02-07T13:34:38.354902+0000","last_clean":"2024-02-07T13:34:38.354902+0000","last_became_active":"2024-02-07T13:34:13.428916+0000","last_became_peered":"2024-02-07T13:34:13.428916+0000","last_unstale":"2024-02-07T13:34:38.354902+0000","last_undegraded":"2024-02-07T13:34:38.354902+0000","last_fullsized":"2024-02-07T13:34:38.354902+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:46:28.881402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469284+0000","last_change":"2024-02-07T13:34:15.436995+0000","last_active":"2024-02-07T13:34:46.469284+0000","last_peered":"2024-02-07T13:34:46.469284+0000","last_clean":"2024-02-07T13:34:46.469284+0000","last_became_active":"2024-02-07T13:34:15.436634+0000","last_became_peered":"2024-02-07T13:34:15.436634+0000","last_unstale":"2024-02-07T13:34:46.469284+0000","last_undegraded":"2024-02-07T13:34:46.469284+0000","last_fullsized":"2024-02-07T13:34:46.469284+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:33:35.094693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355698+0000","last_change":"2024-02-07T13:34:17.447846+0000","last_active":"2024-02-07T13:34:38.355698+0000","last_peered":"2024-02-07T13:34:38.355698+0000","last_clean":"2024-02-07T13:34:38.355698+0000","last_became_active":"2024-02-07T13:34:17.447523+0000","last_became_peered":"2024-02-07T13:34:17.447523+0000","last_unstale":"2024-02-07T13:34:38.355698+0000","last_undegraded":"2024-02-07T13:34:38.355698+0000","last_fullsized":"2024-02-07T13:34:38.355698+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:03:39.234809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355257+0000","last_change":"2024-02-07T13:34:11.424004+0000","last_active":"2024-02-07T13:34:38.355257+0000","last_peered":"2024-02-07T13:34:38.355257+0000","last_clean":"2024-02-07T13:34:38.355257+0000","last_became_active":"2024-02-07T13:34:11.423635+0000","last_became_peered":"2024-02-07T13:34:11.423635+0000","last_unstale":"2024-02-07T13:34:38.355257+0000","last_undegraded":"2024-02-07T13:34:38.355257+0000","last_fullsized":"2024-02-07T13:34:38.355257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:09:16.066441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353048+0000","last_change":"2024-02-07T13:34:13.430111+0000","last_active":"2024-02-07T13:34:38.353048+0000","last_peered":"2024-02-07T13:34:38.353048+0000","last_clean":"2024-02-07T13:34:38.353048+0000","last_became_active":"2024-02-07T13:34:13.429006+0000","last_became_peered":"2024-02-07T13:34:13.429006+0000","last_unstale":"2024-02-07T13:34:38.353048+0000","last_undegraded":"2024-02-07T13:34:38.353048+0000","last_fullsized":"2024-02-07T13:34:38.353048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:41:34.701748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354933+0000","last_change":"2024-02-07T13:34:15.432100+0000","last_active":"2024-02-07T13:34:38.354933+0000","last_peered":"2024-02-07T13:34:38.354933+0000","last_clean":"2024-02-07T13:34:38.354933+0000","last_became_active":"2024-02-07T13:34:15.431821+0000","last_became_peered":"2024-02-07T13:34:15.431821+0000","last_unstale":"2024-02-07T13:34:38.354933+0000","last_undegraded":"2024-02-07T13:34:38.354933+0000","last_fullsized":"2024-02-07T13:34:38.354933+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:04:15.823910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353904+0000","last_change":"2024-02-07T13:34:17.447432+0000","last_active":"2024-02-07T13:34:38.353904+0000","last_peered":"2024-02-07T13:34:38.353904+0000","last_clean":"2024-02-07T13:34:38.353904+0000","last_became_active":"2024-02-07T13:34:17.447023+0000","last_became_peered":"2024-02-07T13:34:17.447023+0000","last_unstale":"2024-02-07T13:34:38.353904+0000","last_undegraded":"2024-02-07T13:34:38.353904+0000","last_fullsized":"2024-02-07T13:34:38.353904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:34:22.049166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469349+0000","last_change":"2024-02-07T13:34:15.437241+0000","last_active":"2024-02-07T13:34:46.469349+0000","last_peered":"2024-02-07T13:34:46.469349+0000","last_clean":"2024-02-07T13:34:46.469349+0000","last_became_active":"2024-02-07T13:34:15.436551+0000","last_became_peered":"2024-02-07T13:34:15.436551+0000","last_unstale":"2024-02-07T13:34:46.469349+0000","last_undegraded":"2024-02-07T13:34:46.469349+0000","last_fullsized":"2024-02-07T13:34:46.469349+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:44:13.780993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352059+0000","last_change":"2024-02-07T13:34:11.425684+0000","last_active":"2024-02-07T13:34:38.352059+0000","last_peered":"2024-02-07T13:34:38.352059+0000","last_clean":"2024-02-07T13:34:38.352059+0000","last_became_active":"2024-02-07T13:34:11.425405+0000","last_became_peered":"2024-02-07T13:34:11.425405+0000","last_unstale":"2024-02-07T13:34:38.352059+0000","last_undegraded":"2024-02-07T13:34:38.352059+0000","last_fullsized":"2024-02-07T13:34:38.352059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:21:53.365086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355412+0000","last_change":"2024-02-07T13:34:13.435291+0000","last_active":"2024-02-07T13:34:38.355412+0000","last_peered":"2024-02-07T13:34:38.355412+0000","last_clean":"2024-02-07T13:34:38.355412+0000","last_became_active":"2024-02-07T13:34:13.434664+0000","last_became_peered":"2024-02-07T13:34:13.434664+0000","last_unstale":"2024-02-07T13:34:38.355412+0000","last_undegraded":"2024-02-07T13:34:38.355412+0000","last_fullsized":"2024-02-07T13:34:38.355412+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:45:38.841078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.357124+0000","last_change":"2024-02-07T13:34:17.450157+0000","last_active":"2024-02-07T13:34:38.357124+0000","last_peered":"2024-02-07T13:34:38.357124+0000","last_clean":"2024-02-07T13:34:38.357124+0000","last_became_active":"2024-02-07T13:34:17.449576+0000","last_became_peered":"2024-02-07T13:34:17.449576+0000","last_unstale":"2024-02-07T13:34:38.357124+0000","last_undegraded":"2024-02-07T13:34:38.357124+0000","last_fullsized":"2024-02-07T13:34:38.357124+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:05:31.746195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353330+0000","last_change":"2024-02-07T13:34:15.436971+0000","last_active":"2024-02-07T13:34:38.353330+0000","last_peered":"2024-02-07T13:34:38.353330+0000","last_clean":"2024-02-07T13:34:38.353330+0000","last_became_active":"2024-02-07T13:34:15.436451+0000","last_became_peered":"2024-02-07T13:34:15.436451+0000","last_unstale":"2024-02-07T13:34:38.353330+0000","last_undegraded":"2024-02-07T13:34:38.353330+0000","last_fullsized":"2024-02-07T13:34:38.353330+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:08:21.213044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350877+0000","last_change":"2024-02-07T13:34:11.420848+0000","last_active":"2024-02-07T13:34:38.350877+0000","last_peered":"2024-02-07T13:34:38.350877+0000","last_clean":"2024-02-07T13:34:38.350877+0000","last_became_active":"2024-02-07T13:34:11.419695+0000","last_became_peered":"2024-02-07T13:34:11.419695+0000","last_unstale":"2024-02-07T13:34:38.350877+0000","last_undegraded":"2024-02-07T13:34:38.350877+0000","last_fullsized":"2024-02-07T13:34:38.350877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:39:49.334893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355152+0000","last_change":"2024-02-07T13:34:13.432434+0000","last_active":"2024-02-07T13:34:38.355152+0000","last_peered":"2024-02-07T13:34:38.355152+0000","last_clean":"2024-02-07T13:34:38.355152+0000","last_became_active":"2024-02-07T13:34:13.431810+0000","last_became_peered":"2024-02-07T13:34:13.431810+0000","last_unstale":"2024-02-07T13:34:38.355152+0000","last_undegraded":"2024-02-07T13:34:38.355152+0000","last_fullsized":"2024-02-07T13:34:38.355152+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:41:44.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356156+0000","last_change":"2024-02-07T13:34:17.449293+0000","last_active":"2024-02-07T13:34:38.356156+0000","last_peered":"2024-02-07T13:34:38.356156+0000","last_clean":"2024-02-07T13:34:38.356156+0000","last_became_active":"2024-02-07T13:34:17.449004+0000","last_became_peered":"2024-02-07T13:34:17.449004+0000","last_unstale":"2024-02-07T13:34:38.356156+0000","last_undegraded":"2024-02-07T13:34:38.356156+0000","last_fullsized":"2024-02-07T13:34:38.356156+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:46:45.518482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355986+0000","last_change":"2024-02-07T13:34:11.419894+0000","last_active":"2024-02-07T13:34:38.355986+0000","last_peered":"2024-02-07T13:34:38.355986+0000","last_clean":"2024-02-07T13:34:38.355986+0000","last_became_active":"2024-02-07T13:34:11.419240+0000","last_became_peered":"2024-02-07T13:34:11.419240+0000","last_unstale":"2024-02-07T13:34:38.355986+0000","last_undegraded":"2024-02-07T13:34:38.355986+0000","last_fullsized":"2024-02-07T13:34:38.355986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:47:26.420102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352123+0000","last_change":"2024-02-07T13:34:13.427521+0000","last_active":"2024-02-07T13:34:38.352123+0000","last_peered":"2024-02-07T13:34:38.352123+0000","last_clean":"2024-02-07T13:34:38.352123+0000","last_became_active":"2024-02-07T13:34:13.427180+0000","last_became_peered":"2024-02-07T13:34:13.427180+0000","last_unstale":"2024-02-07T13:34:38.352123+0000","last_undegraded":"2024-02-07T13:34:38.352123+0000","last_fullsized":"2024-02-07T13:34:38.352123+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:47:00.901391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354966+0000","last_change":"2024-02-07T13:34:15.438867+0000","last_active":"2024-02-07T13:34:38.354966+0000","last_peered":"2024-02-07T13:34:38.354966+0000","last_clean":"2024-02-07T13:34:38.354966+0000","last_became_active":"2024-02-07T13:34:15.438004+0000","last_became_peered":"2024-02-07T13:34:15.438004+0000","last_unstale":"2024-02-07T13:34:38.354966+0000","last_undegraded":"2024-02-07T13:34:38.354966+0000","last_fullsized":"2024-02-07T13:34:38.354966+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:55:45.191362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350484+0000","last_change":"2024-02-07T13:34:17.446334+0000","last_active":"2024-02-07T13:34:38.350484+0000","last_peered":"2024-02-07T13:34:38.350484+0000","last_clean":"2024-02-07T13:34:38.350484+0000","last_became_active":"2024-02-07T13:34:17.445998+0000","last_became_peered":"2024-02-07T13:34:17.445998+0000","last_unstale":"2024-02-07T13:34:38.350484+0000","last_undegraded":"2024-02-07T13:34:38.350484+0000","last_fullsized":"2024-02-07T13:34:38.350484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:50:59.668742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:46.469449+0000","last_change":"2024-02-07T13:34:15.437443+0000","last_active":"2024-02-07T13:34:46.469449+0000","last_peered":"2024-02-07T13:34:46.469449+0000","last_clean":"2024-02-07T13:34:46.469449+0000","last_became_active":"2024-02-07T13:34:15.436781+0000","last_became_peered":"2024-02-07T13:34:15.436781+0000","last_unstale":"2024-02-07T13:34:46.469449+0000","last_undegraded":"2024-02-07T13:34:46.469449+0000","last_fullsized":"2024-02-07T13:34:46.469449+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:16:49.601811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356242+0000","last_change":"2024-02-07T13:34:11.420420+0000","last_active":"2024-02-07T13:34:38.356242+0000","last_peered":"2024-02-07T13:34:38.356242+0000","last_clean":"2024-02-07T13:34:38.356242+0000","last_became_active":"2024-02-07T13:34:11.419430+0000","last_became_peered":"2024-02-07T13:34:11.419430+0000","last_unstale":"2024-02-07T13:34:38.356242+0000","last_undegraded":"2024-02-07T13:34:38.356242+0000","last_fullsized":"2024-02-07T13:34:38.356242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:40:47.899859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352658+0000","last_change":"2024-02-07T13:34:13.429252+0000","last_active":"2024-02-07T13:34:38.352658+0000","last_peered":"2024-02-07T13:34:38.352658+0000","last_clean":"2024-02-07T13:34:38.352658+0000","last_became_active":"2024-02-07T13:34:13.428462+0000","last_became_peered":"2024-02-07T13:34:13.428462+0000","last_unstale":"2024-02-07T13:34:38.352658+0000","last_undegraded":"2024-02-07T13:34:38.352658+0000","last_fullsized":"2024-02-07T13:34:38.352658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:36:31.457455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355461+0000","last_change":"2024-02-07T13:34:17.448542+0000","last_active":"2024-02-07T13:34:38.355461+0000","last_peered":"2024-02-07T13:34:38.355461+0000","last_clean":"2024-02-07T13:34:38.355461+0000","last_became_active":"2024-02-07T13:34:17.447650+0000","last_became_peered":"2024-02-07T13:34:17.447650+0000","last_unstale":"2024-02-07T13:34:38.355461+0000","last_undegraded":"2024-02-07T13:34:38.355461+0000","last_fullsized":"2024-02-07T13:34:38.355461+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:44:11.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355749+0000","last_change":"2024-02-07T13:34:11.424766+0000","last_active":"2024-02-07T13:34:38.355749+0000","last_peered":"2024-02-07T13:34:38.355749+0000","last_clean":"2024-02-07T13:34:38.355749+0000","last_became_active":"2024-02-07T13:34:11.424183+0000","last_became_peered":"2024-02-07T13:34:11.424183+0000","last_unstale":"2024-02-07T13:34:38.355749+0000","last_undegraded":"2024-02-07T13:34:38.355749+0000","last_fullsized":"2024-02-07T13:34:38.355749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:12:57.668280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.353394+0000","last_change":"2024-02-07T13:34:13.429976+0000","last_active":"2024-02-07T13:34:38.353394+0000","last_peered":"2024-02-07T13:34:38.353394+0000","last_clean":"2024-02-07T13:34:38.353394+0000","last_became_active":"2024-02-07T13:34:13.428767+0000","last_became_peered":"2024-02-07T13:34:13.428767+0000","last_unstale":"2024-02-07T13:34:38.353394+0000","last_undegraded":"2024-02-07T13:34:38.353394+0000","last_fullsized":"2024-02-07T13:34:38.353394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:39:49.240386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352504+0000","last_change":"2024-02-07T13:34:15.437645+0000","last_active":"2024-02-07T13:34:38.352504+0000","last_peered":"2024-02-07T13:34:38.352504+0000","last_clean":"2024-02-07T13:34:38.352504+0000","last_became_active":"2024-02-07T13:34:15.436692+0000","last_became_peered":"2024-02-07T13:34:15.436692+0000","last_unstale":"2024-02-07T13:34:38.352504+0000","last_undegraded":"2024-02-07T13:34:38.352504+0000","last_fullsized":"2024-02-07T13:34:38.352504+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:06:56.470469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.349799+0000","last_change":"2024-02-07T13:34:17.444188+0000","last_active":"2024-02-07T13:34:38.349799+0000","last_peered":"2024-02-07T13:34:38.349799+0000","last_clean":"2024-02-07T13:34:38.349799+0000","last_became_active":"2024-02-07T13:34:17.443808+0000","last_became_peered":"2024-02-07T13:34:17.443808+0000","last_unstale":"2024-02-07T13:34:38.349799+0000","last_undegraded":"2024-02-07T13:34:38.349799+0000","last_fullsized":"2024-02-07T13:34:38.349799+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:30:40.258989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.356650+0000","last_change":"2024-02-07T13:34:11.423931+0000","last_active":"2024-02-07T13:34:38.356650+0000","last_peered":"2024-02-07T13:34:38.356650+0000","last_clean":"2024-02-07T13:34:38.356650+0000","last_became_active":"2024-02-07T13:34:11.423643+0000","last_became_peered":"2024-02-07T13:34:11.423643+0000","last_unstale":"2024-02-07T13:34:38.356650+0000","last_undegraded":"2024-02-07T13:34:38.356650+0000","last_fullsized":"2024-02-07T13:34:38.356650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:15:21.761202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355134+0000","last_change":"2024-02-07T13:34:13.428659+0000","last_active":"2024-02-07T13:34:38.355134+0000","last_peered":"2024-02-07T13:34:38.355134+0000","last_clean":"2024-02-07T13:34:38.355134+0000","last_became_active":"2024-02-07T13:34:13.427422+0000","last_became_peered":"2024-02-07T13:34:13.427422+0000","last_unstale":"2024-02-07T13:34:38.355134+0000","last_undegraded":"2024-02-07T13:34:38.355134+0000","last_fullsized":"2024-02-07T13:34:38.355134+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:32:25.128929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.350520+0000","last_change":"2024-02-07T13:34:15.435366+0000","last_active":"2024-02-07T13:34:38.350520+0000","last_peered":"2024-02-07T13:34:38.350520+0000","last_clean":"2024-02-07T13:34:38.350520+0000","last_became_active":"2024-02-07T13:34:15.435045+0000","last_became_peered":"2024-02-07T13:34:15.435045+0000","last_unstale":"2024-02-07T13:34:38.350520+0000","last_undegraded":"2024-02-07T13:34:38.350520+0000","last_fullsized":"2024-02-07T13:34:38.350520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:50:15.285728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352030+0000","last_change":"2024-02-07T13:34:17.447321+0000","last_active":"2024-02-07T13:34:38.352030+0000","last_peered":"2024-02-07T13:34:38.352030+0000","last_clean":"2024-02-07T13:34:38.352030+0000","last_became_active":"2024-02-07T13:34:17.446988+0000","last_became_peered":"2024-02-07T13:34:17.446988+0000","last_unstale":"2024-02-07T13:34:38.352030+0000","last_undegraded":"2024-02-07T13:34:38.352030+0000","last_fullsized":"2024-02-07T13:34:38.352030+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:16.430092+0000","last_clean_scrub_stamp":"2024-02-07T13:34:16.430092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:43:01.522028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.352957+0000","last_change":"2024-02-07T13:34:11.420939+0000","last_active":"2024-02-07T13:34:38.352957+0000","last_peered":"2024-02-07T13:34:38.352957+0000","last_clean":"2024-02-07T13:34:38.352957+0000","last_became_active":"2024-02-07T13:34:11.420072+0000","last_became_peered":"2024-02-07T13:34:11.420072+0000","last_unstale":"2024-02-07T13:34:38.352957+0000","last_undegraded":"2024-02-07T13:34:38.352957+0000","last_fullsized":"2024-02-07T13:34:38.352957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:10.404244+0000","last_clean_scrub_stamp":"2024-02-07T13:34:10.404244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:22:54.546023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.354737+0000","last_change":"2024-02-07T13:34:13.430137+0000","last_active":"2024-02-07T13:34:38.354737+0000","last_peered":"2024-02-07T13:34:38.354737+0000","last_clean":"2024-02-07T13:34:38.354737+0000","last_became_active":"2024-02-07T13:34:13.429300+0000","last_became_peered":"2024-02-07T13:34:13.429300+0000","last_unstale":"2024-02-07T13:34:38.354737+0000","last_undegraded":"2024-02-07T13:34:38.354737+0000","last_fullsized":"2024-02-07T13:34:38.354737+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:12.410497+0000","last_clean_scrub_stamp":"2024-02-07T13:34:12.410497+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:05:04.356701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-07T13:34:38.355380+0000","last_change":"2024-02-07T13:34:15.436747+0000","last_active":"2024-02-07T13:34:38.355380+0000","last_peered":"2024-02-07T13:34:38.355380+0000","last_clean":"2024-02-07T13:34:38.355380+0000","last_became_active":"2024-02-07T13:34:15.436486+0000","last_became_peered":"2024-02-07T13:34:15.436486+0000","last_unstale":"2024-02-07T13:34:38.355380+0000","last_undegraded":"2024-02-07T13:34:38.355380+0000","last_fullsized":"2024-02-07T13:34:38.355380+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:34:14.419039+0000","last_clean_scrub_stamp":"2024-02-07T13:34:14.419039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:08:42.751142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738392,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93704692,"statfs":{"total":95990841344,"available":95953604608,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 7 13:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 7 13:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.371}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-07T13:34:58.821 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-07T13:34:58.821 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-07T13:34:58.821 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-07T13:34:58.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- ceph health --format=json 2024-02-07T13:34:59.468 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:34:59 smithi094 ceph-mon[25576]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:34:59.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:34:59 smithi190 ceph-mon[30713]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:34:59.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:34:59 smithi138 ceph-mon[30535]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:34:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:35:00.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:00 smithi094 ceph-mon[25576]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:00.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:00 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ::ffff:172.21.15.138 - - [07/Feb/2024:13:35:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:35:00.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:00 smithi190 ceph-mon[30713]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:00.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:00 smithi138 ceph-mon[30535]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:02.314 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:02 smithi094 ceph-mon[25576]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:02.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:02 smithi190 ceph-mon[30713]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:02.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:02 smithi138 ceph-mon[30535]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:03.889 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-07T13:35:03.890 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-07T13:35:03.890 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:03 smithi094 podman[56658]: 2024-02-07 13:35:03.849513908 +0000 UTC m=+0.304029001 container create 4012c0c6d40faacb8c1035ebb1f092a176d8ad8ab3b9ba114dcbd64f57d9273e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:35:04.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-mon[25576]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:04.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-mon[25576]: from='client.? 172.21.15.94:0/2749155622' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-07T13:35:04.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:04 smithi190 ceph-mon[30713]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:04.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:04 smithi190 ceph-mon[30713]: from='client.? 172.21.15.94:0/2749155622' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-07T13:35:04.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:04 smithi138 ceph-mon[30535]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:04.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:04 smithi138 ceph-mon[30535]: from='client.? 172.21.15.94:0/2749155622' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-07T13:35:04.666 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-07T13:35:04.666 INFO:tasks.cephadm:Setup complete, yielding 2024-02-07T13:35:04.666 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-07T13:35:04.676 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-02-07T13:35:04.676 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-02-07T13:35:04.898 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 podman[56658]: 2024-02-07 13:35:04.790400711 +0000 UTC m=+1.244915809 container init 4012c0c6d40faacb8c1035ebb1f092a176d8ad8ab3b9ba114dcbd64f57d9273e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:35:04.898 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 podman[56658]: 2024-02-07 13:35:04.796207326 +0000 UTC m=+1.250722411 container start 4012c0c6d40faacb8c1035ebb1f092a176d8ad8ab3b9ba114dcbd64f57d9273e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:35:04.898 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:04.808Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-07T13:35:04.898 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:04.808Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-07T13:35:04.898 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:04.809Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.94 port=9094 2024-02-07T13:35:04.898 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:04.810Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-07T13:35:04.898 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:04.838Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-07T13:35:04.898 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:04.838Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-07T13:35:04.899 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:04.842Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-07T13:35:04.899 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:04.842Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-07T13:35:04.899 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 bash[56658]: 4012c0c6d40faacb8c1035ebb1f092a176d8ad8ab3b9ba114dcbd64f57d9273e 2024-02-07T13:35:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:35:05.181 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:04 smithi094 systemd[1]: Started Ceph alertmanager.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:35:05.506 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:05] ENGINE Bus STOPPING 2024-02-07T13:35:05.506 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-07T13:35:05.506 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:05] ENGINE Bus STOPPED 2024-02-07T13:35:05.506 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:05] ENGINE Bus STARTING 2024-02-07T13:35:05.506 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:05] ENGINE Serving on http://:::9283 2024-02-07T13:35:05.506 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:05] ENGINE Bus STARTED 2024-02-07T13:35:06.122 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.123 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:05 smithi094 ceph-mon[25576]: Deploying daemon grafana.a on smithi138 2024-02-07T13:35:06.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:06.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-07T13:35:06.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:35:06.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:35:06.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:05 smithi190 ceph-mon[30713]: Deploying daemon grafana.a on smithi138 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:35:06.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:35:06.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:06.344 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:05 smithi138 ceph-mon[30535]: Deploying daemon grafana.a on smithi138 2024-02-07T13:35:06.901 INFO:teuthology.orchestra.run.smithi094.stdout:167 167 2024-02-07T13:35:07.263 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:06 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:06.811Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000140343s 2024-02-07T13:35:07.720 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- bash -c 'ceph orch status' 2024-02-07T13:35:07.960 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:35:08.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:07 smithi094 ceph-mon[25576]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:08.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:07 smithi190 ceph-mon[30713]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:08.343 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:07 smithi138 ceph-mon[30535]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:09.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:09 smithi094 ceph-mon[25576]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:09.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:09 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:09.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:09 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:35:09.685 INFO:teuthology.orchestra.run.smithi094.stdout:Backend: cephadm 2024-02-07T13:35:09.685 INFO:teuthology.orchestra.run.smithi094.stdout:Available: Yes 2024-02-07T13:35:09.685 INFO:teuthology.orchestra.run.smithi094.stdout:Paused: No 2024-02-07T13:35:09.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:09 smithi190 ceph-mon[30713]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:09.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:09 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:09.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:09 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:35:09.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:09 smithi138 ceph-mon[30535]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:09.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:09 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:09.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:09 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:35:10.263 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:10 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ::ffff:172.21.15.138 - - [07/Feb/2024:13:35:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:35:10.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-07T13:35:10.577 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:35:10.688 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:10 smithi094 ceph-mon[25576]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:10.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:10 smithi190 ceph-mon[30713]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:10.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:10 smithi138 ceph-mon[30535]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:11.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:11 smithi094 ceph-mon[25576]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:11.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:11 smithi190 ceph-mon[30713]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:11.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:11 smithi138 ceph-mon[30535]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:12.428 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-07T13:35:12.428 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.a smithi094 *:9093,9094 starting - - - - 2024-02-07T13:35:12.428 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.x smithi138 *:8443,8765 running (2m) 31s ago 2m 447M - 19.0.0-1252-g44458cce 79957b372307 dddb105e80c1 2024-02-07T13:35:12.428 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.y smithi094 *:9283,8765 running (4m) 23s ago 4m 481M - 19.0.0-1252-g44458cce 79957b372307 15e2e624d866 2024-02-07T13:35:12.428 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.z smithi190 *:8443,8765 running (2m) 32s ago 2m 449M - 19.0.0-1252-g44458cce 79957b372307 4df78f4d0932 2024-02-07T13:35:12.428 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a smithi094 running (4m) 23s ago 4m 48.8M 2048M 19.0.0-1252-g44458cce 79957b372307 7b74c730dca7 2024-02-07T13:35:12.428 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b smithi138 running (2m) 31s ago 2m 44.3M 2048M 19.0.0-1252-g44458cce 79957b372307 55ff65143798 2024-02-07T13:35:12.429 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c smithi190 running (2m) 32s ago 2m 45.9M 2048M 19.0.0-1252-g44458cce 79957b372307 6bd712409c6b 2024-02-07T13:35:12.429 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (2m) 23s ago 2m 59.0M 15.6G 19.0.0-1252-g44458cce 79957b372307 d38848e55024 2024-02-07T13:35:12.429 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi138 running (105s) 31s ago 104s 63.3M 15.6G 19.0.0-1252-g44458cce 79957b372307 70949210762f 2024-02-07T13:35:12.429 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi190 running (80s) 32s ago 80s 53.6M 16.6G 19.0.0-1252-g44458cce 79957b372307 5631c5cf5559 2024-02-07T13:35:12.429 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.a smithi138 *:9095 running (42s) 31s ago 42s 24.5M - 2.43.0 a07b618ecd1d 98a954e2991d 2024-02-07T13:35:12.429 INFO:teuthology.orchestra.run.smithi094.stdout:rgw.foo.a smithi094 *:80 running (63s) 23s ago 62s 76.5M - 19.0.0-1252-g44458cce 79957b372307 b518cef04ad6 2024-02-07T13:35:13.089 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-07T13:35:13.324 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:35:13.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:13 smithi094 ceph-mon[25576]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:13.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:13 smithi094 ceph-mon[25576]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:13.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:13 smithi190 ceph-mon[30713]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:13.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:13 smithi190 ceph-mon[30713]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:13.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:13 smithi138 ceph-mon[30535]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:13.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:13 smithi138 ceph-mon[30535]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:15.013 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:35:14 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:35:14.813Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003004881s 2024-02-07T13:35:15.195 INFO:teuthology.orchestra.run.smithi094.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-07T13:35:15.196 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager ?:9093,9094 0/1 - 60s smithi094=a;count:1 2024-02-07T13:35:15.196 INFO:teuthology.orchestra.run.smithi094.stdout:grafana ?:3000 0/1 - 57s smithi138=a;count:1 2024-02-07T13:35:15.196 INFO:teuthology.orchestra.run.smithi094.stdout:mgr 3/3 35s ago 2m smithi094=y;smithi138=x;smithi190=z;count:3 2024-02-07T13:35:15.196 INFO:teuthology.orchestra.run.smithi094.stdout:mon 3/3 35s ago 3m smithi094:172.21.15.94=a;smithi138:172.21.15.138=b;smithi190:172.21.15.190=c;count:3 2024-02-07T13:35:15.196 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter ?:9100 0/3 - 63s smithi094=a;smithi138=b;smithi190=c;count:3 2024-02-07T13:35:15.196 INFO:teuthology.orchestra.run.smithi094.stdout:osd 3 35s ago - 2024-02-07T13:35:15.196 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus ?:9095 1/1 34s ago 66s smithi138=a;count:1 2024-02-07T13:35:15.196 INFO:teuthology.orchestra.run.smithi094.stdout:rgw.foo.a ?:80 1/1 25s ago 65s smithi094=foo.a;count:1 2024-02-07T13:35:15.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:15 smithi094 ceph-mon[25576]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:15.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:15 smithi094 ceph-mon[25576]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:15.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:15 smithi190 ceph-mon[30713]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:15.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:15 smithi190 ceph-mon[30713]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:15.822 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-07T13:35:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:15 smithi138 ceph-mon[30535]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:15.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:15 smithi138 ceph-mon[30535]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:16.055 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:35:17.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:17 smithi094 ceph-mon[25576]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:17.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:17 smithi190 ceph-mon[30713]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:17.838 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:17 smithi138 ceph-mon[30535]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:17.887 INFO:teuthology.orchestra.run.smithi094.stdout:HOST ADDR LABELS STATUS 2024-02-07T13:35:17.887 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 172.21.15.94 2024-02-07T13:35:17.887 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 172.21.15.138 2024-02-07T13:35:17.887 INFO:teuthology.orchestra.run.smithi094.stdout:smithi190 172.21.15.190 2024-02-07T13:35:17.888 INFO:teuthology.orchestra.run.smithi094.stdout:3 hosts in cluster 2024-02-07T13:35:18.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:18 smithi094 ceph-mon[25576]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:18.576 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-07T13:35:18.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:18 smithi190 ceph-mon[30713]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:18.807 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:18 smithi138 ceph-mon[30535]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:18.807 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:18 smithi138 systemd[1]: Starting Ceph grafana.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:18.815 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:35:19.093 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:18 smithi138 podman[42811]: 2024-02-07 13:35:18.709919497 +0000 UTC m=+0.016982129 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-07T13:35:19.093 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:18 smithi138 podman[42811]: 2024-02-07 13:35:18.808472019 +0000 UTC m=+0.115534661 container create a3e0fd96e69264a3b02d707b1a532aa33003748ab27ed7bd1aa649b92f40643c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, release=239.1651231664, architecture=x86_64, maintainer=Paul Cuzner , distribution-scope=public, io.openshift.expose-services=, com.redhat.component=ubi8-container, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-02-07T13:35:19.093 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 podman[42811]: 2024-02-07 13:35:19.03560829 +0000 UTC m=+0.342670919 container init a3e0fd96e69264a3b02d707b1a532aa33003748ab27ed7bd1aa649b92f40643c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, name=ubi8, description=Ceph Grafana Container, architecture=x86_64, io.buildah.version=1.32.0, version=8.5, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, release=239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-02-07T13:35:19.094 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 podman[42811]: 2024-02-07 13:35:19.040892145 +0000 UTC m=+0.347954768 container start a3e0fd96e69264a3b02d707b1a532aa33003748ab27ed7bd1aa649b92f40643c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, release=239.1651231664, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, description=Ceph Grafana Container, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, version=8.5, name=ubi8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0) 2024-02-07T13:35:19.379 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 bash[42811]: a3e0fd96e69264a3b02d707b1a532aa33003748ab27ed7bd1aa649b92f40643c 2024-02-07T13:35:19.379 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 systemd[1]: Started Ceph grafana.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196651794Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196885418Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196918959Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196931779Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196943561Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196955241Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196965997Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196977233Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196988016Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-07T13:35:19.380 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.196998482Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.197008328Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.197018616Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.197030346Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=settings t=2024-02-07T13:35:19.197040977Z level=info msg="App mode production" 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=sqlstore t=2024-02-07T13:35:19.19710767Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=sqlstore t=2024-02-07T13:35:19.197132835Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.198269232Z level=info msg="Starting DB migrations" 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.198365983Z level=info msg="Executing migration" id="create migration_log table" 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.198918001Z level=info msg="Executing migration" id="create user table" 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.19939807Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-07T13:35:19.381 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.199862183Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.200392542Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.200917214Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.201302924Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.201973839Z level=info msg="Executing migration" id="create user table v2" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.202368183Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.202749288Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.203225807Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.203649717Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.204049771Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.204498658Z level=info msg="Executing migration" id="Update user table charset" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.204797974Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.205288164Z level=info msg="Executing migration" id="Add missing user data" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.205683658Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-07T13:35:19.382 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.206137521Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-07T13:35:19.383 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.206603999Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-07T13:35:19.383 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.207083189Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.208230298Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.20877042Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.209218775Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.209604114Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.210021745Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.210916688Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.21159283Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.212387999Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.212808256Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.213244682Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.214124662Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-07T13:35:19.384 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.215018246Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.21582657Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.216616813Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.217407025Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.220198585Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.222940109Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.225565505Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.228139155Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.228506148Z level=info msg="Executing migration" id="create star table" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.231167875Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.233712456Z level=info msg="Executing migration" id="create org table v1" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.236577805Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-07T13:35:19.385 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.238944322Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.241570297Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.24408896Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.246493513Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.246966134Z level=info msg="Executing migration" id="Update org table charset" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.249717162Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.250128857Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.250970403Z level=info msg="Executing migration" id="create dashboard table" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.251910516Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.252525035Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.253374364Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.254241277Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.255140249Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.255847991Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-07T13:35:19.386 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.258439678Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.261194949Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.263647083Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.266217401Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.268825792Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.271558091Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.274248542Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.276774743Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.279631793Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.282243757Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.284767258Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.287411975Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.289975387Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-07T13:35:19.387 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.292558886Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.295061173Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.297706941Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.30035038Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.302746431Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.305337505Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.308074264Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.310850036Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.313296885Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.315932662Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.318418579Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.321192968Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.323451465Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.326229235Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-07T13:35:19.388 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.328723819Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.331288245Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.333878607Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.336166419Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.338800447Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.34143965Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.343969378Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.346569004Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.347399517Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.348056136Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.348920797Z level=info msg="Executing migration" id="create data_source table" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.349770301Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.350463547Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.351319639Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-07T13:35:19.389 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.35209032Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.352923638Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.354044217Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.354842411Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.355735183Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.356455Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.357208726Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.357788868Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.358386142Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.359274409Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.360022224Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.360838638Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.361614296Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.362366794Z level=info msg="Executing migration" id="Add uid column" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.362985581Z level=info msg="Executing migration" id="Update uid value" 2024-02-07T13:35:19.390 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.363322595Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.364201439Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.364753121Z level=info msg="Executing migration" id="create api_key table" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.365187156Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.366029967Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.367003017Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.367652988Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.368471765Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.369225391Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.370077152Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.371117449Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.371955109Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.372780823Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.373712741Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-07T13:35:19.391 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.374201738Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-07T13:35:19.392 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.377057122Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-07T13:35:19.392 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.379697951Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-07T13:35:19.392 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.382231154Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-07T13:35:19.392 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.382801575Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-07T13:35:19.632 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.385405202Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-07T13:35:19.632 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.3880426Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.390521427Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.391142269Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.39367543Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.396265956Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.398947389Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.399379513Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.402103261Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.404744998Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.407371305Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.407693128Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-07T13:35:19.633 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.410184868Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-07T13:35:19.634 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.412685931Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-07T13:35:19.634 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.41528761Z level=info msg="Executing migration" id="create quota table v1" 2024-02-07T13:35:19.634 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.415837732Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-07T13:35:19.634 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.418395318Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-07T13:35:19.634 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.421203386Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-07T13:35:19.635 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.423784636Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-07T13:35:19.635 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.424263449Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.426980636Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.42742839Z level=info msg="Executing migration" id="create session table" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.429477993Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.431950509Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.432278284Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.434878622Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.437540374Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.440028345Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.440367838Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-07T13:35:19.636 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.442881004Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-07T13:35:19.637 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.443218511Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-07T13:35:19.637 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.443721036Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-07T13:35:19.637 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.444634671Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.445442182Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.446401923Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.447103139Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.448039017Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.44865617Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.449504994Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.450331247Z level=info msg="Executing migration" id="create alert table v1" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.451076231Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.451534006Z level=info msg="Executing migration" id="add index alert state" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.452306851Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.453228124Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-07T13:35:19.638 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.453943289Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.454395511Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.455233419Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.456906298Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.457348989Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.458118237Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.458443321Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.458785851Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.459204495Z level=info msg="Executing migration" id="Add column is_default" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.46173543Z level=info msg="Executing migration" id="Add column frequency" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.462414683Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.463093285Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-07T13:35:19.639 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.463713719Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.464139437Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.464419179Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.464671738Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.465045644Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.465444468Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.465790662Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.466216097Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.468382466Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.469080387Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.471803658Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.474406086Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-07T13:35:19.640 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.476950312Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.477348827Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.480066055Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.480455744Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.482554889Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.483170732Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.48505825Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.485540172Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.488067576Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.490466449Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.49317985Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-07T13:35:19.641 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.49365144Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.496159183Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.498614586Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.501174071Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.501661514Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.50242812Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.503380651Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.504025872Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.504469239Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.506003Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.506787715Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.507815626Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-07T13:35:19.642 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.508197626Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.509002009Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.509939645Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.510815013Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.511542276Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.512314897Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.513195124Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.513849583Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.514569879Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.515379741Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.51621606Z level=info msg="Executing migration" id="Move region to single row" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.517001225Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-07T13:35:19.643 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.517438415Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.518346586Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.518859614Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.519308239Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.52010238Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.520559163Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.520839564Z level=info msg="Executing migration" id="create test_data table" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.521276744Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.52168106Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.522647398Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.523367526Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-07T13:35:19.644 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.523689692Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.524504041Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.524823534Z level=info msg="Executing migration" id="create team table" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.525394982Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.526173878Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.526627645Z level=info msg="Executing migration" id="create team member table" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.529325806Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.531990187Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.534646164Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.535189621Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.53778877Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.540328701Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.543120644Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-07T13:35:19.645 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.543549181Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.546369631Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.548992784Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.549592232Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.550035789Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.552601442Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.553074856Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.553517284Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.553878582Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.554176176Z level=info msg="Executing migration" id="create tag table" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.554540492Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.554973891Z level=info msg="Executing migration" id="create login attempt table" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.555424326Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.555834399Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-07T13:35:19.646 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.556262148Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.558083931Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.558461057Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.558870014Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.559190269Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.559517786Z level=info msg="Executing migration" id="create user auth table" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.559858706Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.560313764Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.560573975Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.561384448Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-07T13:35:19.647 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.56213912Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-07T13:35:19.648 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.562874935Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-07T13:35:19.648 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.563647028Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-07T13:35:19.648 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.564071225Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-07T13:35:19.648 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.564789972Z level=info msg="Executing migration" id="create server_lock table" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.565184393Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.565610276Z level=info msg="Executing migration" id="create user auth token table" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.566020311Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.569064001Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.569596224Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.570003467Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.57075249Z level=info msg="Executing migration" id="create cache_data table" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.571170334Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.571588934Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.572006213Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.572458276Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-07T13:35:19.649 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.572722113Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.573021089Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.57470473Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.57703216Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.579681608Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.579998127Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.582542409Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.585191319Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.587637937Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.590194221Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.592826809Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-07T13:35:19.650 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.595567081Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.597876204Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.600392697Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.603111109Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.605687703Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.608312537Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.610851766Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.613267269Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.615977599Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.618663595Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.621157133Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.623715554Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.626174126Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-07T13:35:19.651 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.629625861Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-07T13:35:19.652 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.632934405Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-07T13:35:19.652 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.63371242Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-07T13:35:19.652 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.634533641Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-07T13:35:19.652 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.635421866Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-07T13:35:19.652 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.636303594Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-07T13:35:19.652 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.63730693Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-07T13:35:19.652 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.637969603Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.638850265Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.639634745Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.640485251Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.641321734Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.642143569Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.642972424Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.645532498Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.648148413Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.650734249Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.653261937Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-07T13:35:19.883 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.656021425Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-07T13:35:19.884 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.658626253Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-07T13:35:19.884 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.661186586Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-07T13:35:19.884 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.66393575Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-07T13:35:19.884 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.66663748Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-07T13:35:19.884 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.669325145Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-07T13:35:19.884 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.671894967Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-07T13:35:19.884 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.674503592Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-07T13:35:19.885 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.677338207Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-07T13:35:19.885 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.680076185Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-07T13:35:19.885 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.68260565Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-07T13:35:19.885 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.6851721Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-07T13:35:19.885 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.68794507Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-07T13:35:19.885 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.690372517Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.692943941Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.695393599Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.698167876Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.700872216Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.703285894Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.705775337Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.708533423Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.71102139Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.71383843Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.71612197Z level=info msg="Executing migration" id="create alert_image table" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.718703382Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.721342728Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.723753169Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.726404701Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-07T13:35:19.886 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.72903488Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.729154461Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.731456489Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.734037623Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.734882618Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.735001709Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.735108978Z level=info msg="alerts found to migrate" alerts=0 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.735670064Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.736478618Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.737259123Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.738159403Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.738869045Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.739737835Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.740560217Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-07T13:35:19.887 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.741369576Z level=info msg="Executing migration" id="create data_keys table" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.742194268Z level=info msg="Executing migration" id="create secrets table" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.743126008Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.747610841Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.748547422Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.749472116Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.753761302Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.757956735Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.758783587Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.759715551Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.76037533Z level=info msg="Executing migration" id="create permission table" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.761163455Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.762117455Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.763061648Z level=info msg="Executing migration" id="create role table" 2024-02-07T13:35:19.888 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.763669466Z level=info msg="Executing migration" id="add column display_name" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.764645045Z level=info msg="Executing migration" id="add column group_name" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.765665013Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.766487558Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.767441871Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.768111687Z level=info msg="Executing migration" id="create team role table" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.768955982Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.76987191Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.770586248Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.771516058Z level=info msg="Executing migration" id="create user role table" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.772188962Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.773118329Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.773798941Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.776465191Z level=info msg="Executing migration" id="create builtin role table" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.779187258Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-07T13:35:19.889 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.781980984Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.78456739Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.787273017Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.789718999Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.792433888Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.794844579Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.797653511Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.800252641Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.802825845Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.805482754Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.807986305Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.810653371Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.813017892Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-07T13:35:19.890 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.815758035Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.818374861Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.820941728Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.823547458Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.826271795Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.829041842Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.831581162Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.833970487Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.836726051Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.839373678Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.841940828Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.844561716Z level=info msg="Executing migration" id="add correlation config column" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.847213449Z level=info msg="Executing migration" id="create entity_events table" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.849662759Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.852075138Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-07T13:35:19.891 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.852187402Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-07T13:35:19.892 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.854619887Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-07T13:35:19.892 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.854725426Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-07T13:35:19.892 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.857254338Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-07T13:35:19.892 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.859804241Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-07T13:35:19.892 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.862652166Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-07T13:35:19.892 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.865011095Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-07T13:35:19.892 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.865700762Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.866621464Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.867345467Z level=info msg="Executing migration" id="Drop public config table" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.868148Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.869063082Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.86974932Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.87059802Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.871452712Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.874741539Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.875749428Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.876752863Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.877536261Z level=info msg="Executing migration" id="add share column" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.878552961Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.879366979Z level=info msg="Executing migration" id="create file table" 2024-02-07T13:35:19.893 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.880132554Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-07T13:35:19.894 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.880885906Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-07T13:35:19.894 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.881632Z level=info msg="Executing migration" id="create file_meta table" 2024-02-07T13:35:19.894 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.882010548Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-07T13:35:19.894 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.883048369Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-07T13:35:19.894 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.883640545Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-07T13:35:19.894 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.884481424Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-07T13:35:19.894 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.88529922Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-07T13:35:19.894 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.886066175Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-07T13:35:20.182 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.887187411Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-07T13:35:20.182 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.888029886Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-07T13:35:20.182 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.888857163Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-07T13:35:20.183 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.889633283Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-07T13:35:20.183 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.890392602Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-07T13:35:20.183 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.89078826Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-07T13:35:20.183 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.89183313Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-07T13:35:20.183 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.892895051Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-07T13:35:20.183 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.893703874Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-07T13:35:20.183 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.903539532Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-07T13:35:20.183 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.904408287Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.905233226Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.908678043Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.909467222Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.912111876Z level=info msg="Executing migration" id="create folder table" 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.914845395Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.917609207Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.920358147Z level=info msg="Executing migration" id="Update folder title length" 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.922885197Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=migrator t=2024-02-07T13:35:19.925351102Z level=info msg="migrations completed" performed=485 skipped=0 duration=727.017251ms 2024-02-07T13:35:20.184 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=sqlstore t=2024-02-07T13:35:19.92578605Z level=info msg="Created default organization" 2024-02-07T13:35:20.185 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:19 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=plugin.loader t=2024-02-07T13:35:19.980134462Z level=info msg="Plugin registered" pluginID=input 2024-02-07T13:35:20.185 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=plugin.loader t=2024-02-07T13:35:20.007780664Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-07T13:35:20.185 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=plugin.loader t=2024-02-07T13:35:20.007931302Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-07T13:35:20.185 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=secrets t=2024-02-07T13:35:20.008098075Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-07T13:35:20.185 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=query_data t=2024-02-07T13:35:20.010221843Z level=info msg="Query Service initialization" 2024-02-07T13:35:20.185 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=live.push_http t=2024-02-07T13:35:20.012639128Z level=info msg="Live Push Gateway initialization" 2024-02-07T13:35:20.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:20 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ::ffff:172.21.15.138 - - [07/Feb/2024:13:35:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:35:20.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:20 smithi094 ceph-mon[25576]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:20.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:20 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:20 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:20 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:20 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:20 smithi094 ceph-mon[25576]: Deploying daemon node-exporter.a on smithi094 2024-02-07T13:35:20.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:20 smithi190 ceph-mon[30713]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:20.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:20 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:20 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:20 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:20 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:20 smithi190 ceph-mon[30713]: Deploying daemon node-exporter.a on smithi094 2024-02-07T13:35:20.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-mon[30535]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:20.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:20.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-mon[30535]: Deploying daemon node-exporter.a on smithi094 2024-02-07T13:35:20.651 INFO:teuthology.orchestra.run.smithi094.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-07T13:35:20.651 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme1n1 ssd Linux_f8d5754f0aee206d036f 89.4G Yes 2m ago 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme2n1 ssd Linux_0d5dcead4e7bbc2f2a0d 89.4G Yes 2m ago 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme3n1 ssd Linux_2b13d56059c566e9829b 89.4G Yes 2m ago 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme4n1 ssd Linux_bdf1def68e01760597d2 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 105s ago Has a FileSystem, LVM detected 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 /dev/nvme1n1 ssd Linux_6091bc1dc322f79ae5a6 89.4G Yes 105s ago 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 /dev/nvme2n1 ssd Linux_138f1f7bbf60300bb1e0 89.4G Yes 105s ago 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 /dev/nvme3n1 ssd Linux_58fcba7ac1e6ecacbaa1 89.4G Yes 105s ago 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 /dev/nvme4n1 ssd Linux_e70c19876b5f9eb8f13c 89.4G No 105s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 81s ago Has a FileSystem, LVM detected 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi190 /dev/nvme1n1 ssd Linux_9cbc534b67f706b67652 89.4G Yes 81s ago 2024-02-07T13:35:20.652 INFO:teuthology.orchestra.run.smithi094.stdout:smithi190 /dev/nvme2n1 ssd Linux_cccddb2c9e925480777e 89.4G Yes 81s ago 2024-02-07T13:35:20.653 INFO:teuthology.orchestra.run.smithi094.stdout:smithi190 /dev/nvme3n1 ssd Linux_56b8d14c79c131392f9f 89.4G Yes 81s ago 2024-02-07T13:35:20.653 INFO:teuthology.orchestra.run.smithi094.stdout:smithi190 /dev/nvme4n1 ssd Linux_8c76b88805ed3432cc76 89.4G No 81s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-07T13:35:21.093 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=infra.usagestats.collector t=2024-02-07T13:35:20.619085338Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-07T13:35:21.093 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=provisioning.datasources t=2024-02-07T13:35:20.620070126Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-07T13:35:21.093 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=provisioning.datasources t=2024-02-07T13:35:20.630916669Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-07T13:35:21.093 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=provisioning.alerting t=2024-02-07T13:35:20.641769583Z level=info msg="starting to provision alerting" 2024-02-07T13:35:21.094 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=provisioning.alerting t=2024-02-07T13:35:20.641788197Z level=info msg="finished to provision alerting" 2024-02-07T13:35:21.094 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=grafanaStorageLogger t=2024-02-07T13:35:20.642045092Z level=info msg="storage starting" 2024-02-07T13:35:21.094 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=ngalert.state.manager t=2024-02-07T13:35:20.642224545Z level=info msg="Warming state cache for startup" 2024-02-07T13:35:21.094 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=http.server t=2024-02-07T13:35:20.643792281Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-07T13:35:21.094 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=ngalert.state.manager t=2024-02-07T13:35:20.645449453Z level=info msg="State cache has been initialized" states=0 duration=3.223758ms 2024-02-07T13:35:21.094 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=ngalert.multiorg.alertmanager t=2024-02-07T13:35:20.64550694Z level=info msg="starting MultiOrg Alertmanager" 2024-02-07T13:35:21.094 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=ticker t=2024-02-07T13:35:20.645516786Z level=info msg=starting first_tick=2024-02-07T13:35:30Z 2024-02-07T13:35:21.094 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:35:20 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a[42826]: logger=sqlstore.transactions t=2024-02-07T13:35:20.661365132Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-07T13:35:21.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls --format yaml' 2024-02-07T13:35:21.645 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:35:21.763 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:21 smithi094 bash[58243]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-07T13:35:22.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:22 smithi094 ceph-mon[25576]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:22.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:22 smithi094 ceph-mon[25576]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:22.514 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:22 smithi094 bash[58243]: Getting image source signatures 2024-02-07T13:35:22.514 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:22 smithi094 bash[58243]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-07T13:35:22.514 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:22 smithi094 bash[58243]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-07T13:35:22.514 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:22 smithi094 bash[58243]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-07T13:35:22.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:22 smithi190 ceph-mon[30713]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:22.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:22 smithi190 ceph-mon[30713]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:22.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:22 smithi138 ceph-mon[30535]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:22.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:22 smithi138 ceph-mon[30535]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:35:24.199 INFO:teuthology.orchestra.run.smithi094.stdout:service_type: alertmanager 2024-02-07T13:35:24.199 INFO:teuthology.orchestra.run.smithi094.stdout:service_name: alertmanager 2024-02-07T13:35:24.199 INFO:teuthology.orchestra.run.smithi094.stdout:placement: 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: count: 1 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: hosts: 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi094=a 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout:status: 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: created: '2024-02-07T13:34:14.487191Z' 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: ports: 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: - 9093 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: - 9094 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: running: 0 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: size: 1 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout:events: 2024-02-07T13:35:24.200 INFO:teuthology.orchestra.run.smithi094.stdout:- 2024-02-07T13:35:05.010684Z service:alertmanager [INFO] "service was created" 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout:--- 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout:service_type: grafana 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout:service_name: grafana 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout:placement: 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: count: 1 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: hosts: 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi138=a 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout:spec: 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: anonymous_access: true 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: protocol: https 2024-02-07T13:35:24.201 INFO:teuthology.orchestra.run.smithi094.stdout:status: 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: created: '2024-02-07T13:34:17.231298Z' 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: ports: 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: - 3000 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: running: 0 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: size: 1 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout:events: 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout:- 2024-02-07T13:35:19.235575Z service:grafana [INFO] "service was created" 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout:--- 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout:service_type: mgr 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout:service_name: mgr 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout:placement: 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: count: 3 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: hosts: 2024-02-07T13:35:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi094=y 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi138=x 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi190=z 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout:status: 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: created: '2024-02-07T13:32:38.684943Z' 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: last_refresh: '2024-02-07T13:34:40.002295Z' 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: running: 3 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: size: 3 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout:--- 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout:service_type: mon 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout:service_name: mon 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout:placement: 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: count: 3 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: hosts: 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi094:172.21.15.94=a 2024-02-07T13:35:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi138:172.21.15.138=b 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi190:172.21.15.190=c 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout:status: 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: created: '2024-02-07T13:32:12.860094Z' 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: last_refresh: '2024-02-07T13:34:40.002169Z' 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: running: 3 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: size: 3 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout:--- 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout:service_type: node-exporter 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout:service_name: node-exporter 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout:placement: 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: count: 3 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: hosts: 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi094=a 2024-02-07T13:35:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi138=b 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi190=c 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout:status: 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: created: '2024-02-07T13:34:11.642373Z' 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: ports: 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: - 9100 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: running: 0 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: size: 3 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout:--- 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout:service_type: osd 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout:service_name: osd 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout:unmanaged: true 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout:spec: 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: filter_logic: AND 2024-02-07T13:35:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: objectstore: bluestore 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout:status: 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: container_image_id: 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: last_refresh: '2024-02-07T13:34:40.002378Z' 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: running: 3 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: size: 3 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout:--- 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout:service_type: prometheus 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout:service_name: prometheus 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout:placement: 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: count: 1 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: hosts: 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi138=a 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout:status: 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: created: '2024-02-07T13:34:09.051147Z' 2024-02-07T13:35:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: last_refresh: '2024-02-07T13:34:40.860425Z' 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: ports: 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: - 9095 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: running: 1 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: size: 1 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout:--- 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout:service_type: rgw 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout:service_id: foo.a 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout:service_name: rgw.foo.a 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout:placement: 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: count: 1 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: hosts: 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: - smithi094=foo.a 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout:status: 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: created: '2024-02-07T13:34:09.789682Z' 2024-02-07T13:35:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: last_refresh: '2024-02-07T13:34:49.223216Z' 2024-02-07T13:35:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: ports: 2024-02-07T13:35:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: - 80 2024-02-07T13:35:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: running: 1 2024-02-07T13:35:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: size: 1 2024-02-07T13:35:24.233 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:24 smithi094 ceph-mon[25576]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:24.233 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:24 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:24.233 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:24 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:35:24.233 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:24 smithi094 bash[58243]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-07T13:35:24.513 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:24 smithi094 bash[58243]: Writing manifest to image destination 2024-02-07T13:35:24.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:24 smithi190 ceph-mon[30713]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:24.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:24 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:24.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:24 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:35:24.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:24 smithi138 ceph-mon[30535]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:24.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:24 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:24.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:24 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:35:24.900 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:24 smithi094 podman[58243]: 2024-02-07 13:35:24.78190042 +0000 UTC m=+3.293533879 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-07T13:35:25.269 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-02-07T13:35:25.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-mon[25576]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-07T13:35:25.513 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 podman[58243]: 2024-02-07 13:35:25.418102824 +0000 UTC m=+3.929736283 container create 4ba4235945c53108f757cf81e0ec2087531d6f25f4d439dff87de85be34954f3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:35:25.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:25 smithi190 ceph-mon[30713]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-07T13:35:25.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:25 smithi138 ceph-mon[30535]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-07T13:35:25.794 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:35:25.921 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 podman[58243]: 2024-02-07 13:35:25.670641465 +0000 UTC m=+4.182274923 container init 4ba4235945c53108f757cf81e0ec2087531d6f25f4d439dff87de85be34954f3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:35:25.921 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 podman[58243]: 2024-02-07 13:35:25.67597683 +0000 UTC m=+4.187610289 container start 4ba4235945c53108f757cf81e0ec2087531d6f25f4d439dff87de85be34954f3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:35:25.921 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.679Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.679Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.680Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.680Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.680Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.680Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=arp 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-07T13:35:25.922 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-07T13:35:25.923 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-07T13:35:25.923 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-07T13:35:25.923 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=edac 2024-02-07T13:35:25.923 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-07T13:35:25.923 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-07T13:35:25.923 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-07T13:35:25.923 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-07T13:35:25.923 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=os 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-07T13:35:25.924 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=stat 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=time 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=uname 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-07T13:35:25.925 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-07T13:35:25.926 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-07T13:35:25.926 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a[58527]: ts=2024-02-07T13:35:25.681Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-07T13:35:25.926 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 bash[58243]: 4ba4235945c53108f757cf81e0ec2087531d6f25f4d439dff87de85be34954f3 2024-02-07T13:35:25.926 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:35:25 smithi094 systemd[1]: Started Ceph node-exporter.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:35:26.466 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:26 smithi138 ceph-mon[30535]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:26.467 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:26 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:26.467 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:26 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:26.467 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:26 smithi138 ceph-mon[30535]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:26.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:26 smithi094 ceph-mon[25576]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:26.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:26 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:26.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:26 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:26.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:26 smithi094 ceph-mon[25576]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:26.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:26 smithi190 ceph-mon[30713]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:26.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:26 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:26.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:26 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:26.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:26 smithi190 ceph-mon[30713]: from='mgr.14415 172.21.15.94:0/2975951450' entity='mgr.y' 2024-02-07T13:35:27.093 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:27 smithi138 bash[43098]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-07T13:35:27.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:27 smithi094 ceph-mon[25576]: Deploying daemon node-exporter.b on smithi138 2024-02-07T13:35:27.458 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:27 smithi138 ceph-mon[30535]: Deploying daemon node-exporter.b on smithi138 2024-02-07T13:35:27.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:27 smithi190 ceph-mon[30713]: Deploying daemon node-exporter.b on smithi138 2024-02-07T13:35:27.800 INFO:teuthology.orchestra.run.smithi094.stdout:osd 3 47s ago - 2024-02-07T13:35:27.842 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:27 smithi138 bash[43098]: Getting image source signatures 2024-02-07T13:35:27.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:27 smithi138 bash[43098]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-07T13:35:27.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:27 smithi138 bash[43098]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-07T13:35:27.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:27 smithi138 bash[43098]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-07T13:35:28.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:28 smithi094 ceph-mon[25576]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:28.508 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-07T13:35:28.518 INFO:tasks.cephadm:Teardown begin 2024-02-07T13:35:28.519 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:35:28.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:28 smithi190 ceph-mon[30713]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:28.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:35:28.576 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:28 smithi138 ceph-mon[30535]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:35:28.581 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:35:28.616 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-07T13:35:28.617 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-07T13:35:28.632 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-07T13:35:28.646 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-07T13:35:28.674 INFO:tasks.cephadm:Stopping all daemons... 2024-02-07T13:35:28.674 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-07T13:35:28.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.a 2024-02-07T13:35:28.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:28 smithi094 systemd[1]: Stopping Ceph mon.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:28.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:28 smithi138 bash[43098]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-07T13:35:29.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:28 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a[25553]: 2024-02-07T13:35:28.959+0000 7f711d087700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:35:29.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:28 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a[25553]: 2024-02-07T13:35:28.959+0000 7f711d087700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-07T13:35:29.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:29 smithi094 podman[58815]: 2024-02-07 13:35:29.164051709 +0000 UTC m=+0.306616723 container died 7b74c730dca77210376dcd0f0f19fdf21895bc0de2635d878720bfafd36da988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-07T13:35:29.264 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:28 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:28] ENGINE Bus STOPPING 2024-02-07T13:35:29.264 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:28 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:28] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-07T13:35:29.264 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:28 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:28] ENGINE Bus STOPPED 2024-02-07T13:35:29.264 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:28 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:28] ENGINE Bus STARTING 2024-02-07T13:35:29.264 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:29 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:29] ENGINE Serving on http://:::9283 2024-02-07T13:35:29.264 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:29 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:29] ENGINE Bus STARTED 2024-02-07T13:35:29.264 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:28 smithi138 bash[43098]: Writing manifest to image destination 2024-02-07T13:35:29.566 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 podman[43098]: 2024-02-07 13:35:29.199427009 +0000 UTC m=+2.258392740 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-07T13:35:29.566 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 podman[43098]: 2024-02-07 13:35:29.289480929 +0000 UTC m=+2.348446651 container create eb85e60947b4d4e9495b762cba3db0b79d46b8d523c8723b6152ef961ab389c3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:35:29.566 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 podman[43098]: 2024-02-07 13:35:29.491573718 +0000 UTC m=+2.550539439 container init eb85e60947b4d4e9495b762cba3db0b79d46b8d523c8723b6152ef961ab389c3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:35:29.566 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 podman[43098]: 2024-02-07 13:35:29.496731309 +0000 UTC m=+2.555697033 container start eb85e60947b4d4e9495b762cba3db0b79d46b8d523c8723b6152ef961ab389c3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.500Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.500Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.501Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.501Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.501Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.501Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=arp 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-07T13:35:29.567 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-07T13:35:29.568 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-07T13:35:29.568 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-07T13:35:29.568 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-07T13:35:29.568 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-07T13:35:29.568 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-07T13:35:29.568 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=edac 2024-02-07T13:35:29.568 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-07T13:35:29.568 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-07T13:35:29.568 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-07T13:35:29.569 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=os 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=stat 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=time 2024-02-07T13:35:29.570 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-07T13:35:29.571 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=uname 2024-02-07T13:35:29.571 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-07T13:35:29.571 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-07T13:35:29.571 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-07T13:35:29.571 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-07T13:35:29.571 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b[43182]: ts=2024-02-07T13:35:29.502Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-07T13:35:29.571 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 bash[43098]: eb85e60947b4d4e9495b762cba3db0b79d46b8d523c8723b6152ef961ab389c3 2024-02-07T13:35:29.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:29 smithi094 podman[58815]: 2024-02-07 13:35:29.632805717 +0000 UTC m=+0.775370723 container cleanup 7b74c730dca77210376dcd0f0f19fdf21895bc0de2635d878720bfafd36da988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:35:29.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 07 13:35:29 smithi094 bash[58815]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-a 2024-02-07T13:35:29.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:35:29 smithi138 systemd[1]: Started Ceph node-exporter.b for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:35:30.055 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.a.service' 2024-02-07T13:35:30.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:35:30.104 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-07T13:35:30.104 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-07T13:35:30.104 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.b 2024-02-07T13:35:30.422 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:30 smithi138 systemd[1]: Stopping Ceph mon.b for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:30.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: ::ffff:172.21.15.138 - - [07/Feb/2024:13:35:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:35:30.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:30] ENGINE Bus STOPPING 2024-02-07T13:35:30.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:30] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-07T13:35:30.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:30] ENGINE Bus STOPPED 2024-02-07T13:35:30.513 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:30] ENGINE Bus STARTING 2024-02-07T13:35:30.737 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-b[30512]: 2024-02-07T13:35:30.421+0000 7f3126784700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:35:30.737 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:30 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-b[30512]: 2024-02-07T13:35:30.421+0000 7f3126784700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-07T13:35:30.737 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:30 smithi138 podman[43260]: 2024-02-07 13:35:30.494094174 +0000 UTC m=+0.198302374 container died 55ff65143798347d60a14622d12a6eb029bba3723072585c4dbcc28742ce49c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-b, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:35:31.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:30] ENGINE Serving on http://:::9283 2024-02-07T13:35:31.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:30 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: [07/Feb/2024:13:35:30] ENGINE Bus STARTED 2024-02-07T13:35:31.070 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:30 smithi138 podman[43260]: 2024-02-07 13:35:30.894821146 +0000 UTC m=+0.599029386 container cleanup 55ff65143798347d60a14622d12a6eb029bba3723072585c4dbcc28742ce49c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-b, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.2) 2024-02-07T13:35:31.070 INFO:journalctl@ceph.mon.b.smithi138.stdout:Feb 07 13:35:30 smithi138 bash[43260]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-b 2024-02-07T13:35:31.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.b.service' 2024-02-07T13:35:31.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:35:31.327 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-07T13:35:31.327 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-07T13:35:31.328 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.c 2024-02-07T13:35:31.634 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:31 smithi190 systemd[1]: Stopping Ceph mon.c for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:31 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-c[30678]: 2024-02-07T13:35:31.633+0000 7f02db294700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:35:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:31 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-c[30678]: 2024-02-07T13:35:31.633+0000 7f02db294700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-07T13:35:32.030 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:31 smithi190 podman[43683]: 2024-02-07 13:35:31.858542682 +0000 UTC m=+0.337574523 container died 6bd712409c6ba66ae92b8972ba24261f1e288f5df0ae3fc65e7442467c0fc3e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-c, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-07T13:35:32.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:32 smithi190 podman[43683]: 2024-02-07 13:35:32.322254819 +0000 UTC m=+0.801286616 container cleanup 6bd712409c6ba66ae92b8972ba24261f1e288f5df0ae3fc65e7442467c0fc3e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.license=GPLv2) 2024-02-07T13:35:32.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Feb 07 13:35:32 smithi190 bash[43683]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mon-c 2024-02-07T13:35:32.684 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mon.c.service' 2024-02-07T13:35:32.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:35:32.731 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-07T13:35:32.731 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-07T13:35:32.731 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.y 2024-02-07T13:35:33.013 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:32 smithi094 systemd[1]: Stopping Ceph mgr.y for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:33.347 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:33 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:35:33.056+0000 7fecf60d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:35:33.347 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:33 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y[25832]: 2024-02-07T13:35:33.056+0000 7fecf60d4700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-07T13:35:33.347 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:33 smithi094 podman[58970]: 2024-02-07 13:35:33.088314542 +0000 UTC m=+0.163840032 container died 15e2e624d8661c76f4ccd17ffec73a01317f4be9898327f885257ecaabe7658e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-07T13:35:33.705 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:33 smithi094 podman[58970]: 2024-02-07 13:35:33.521082808 +0000 UTC m=+0.596608352 container cleanup 15e2e624d8661c76f4ccd17ffec73a01317f4be9898327f885257ecaabe7658e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-07T13:35:33.705 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:33 smithi094 bash[58970]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y 2024-02-07T13:35:33.941 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.y.service' 2024-02-07T13:35:33.964 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:33 smithi094 podman[58988]: 2024-02-07 13:35:33.706367823 +0000 UTC m=+0.619635619 container remove 15e2e624d8661c76f4ccd17ffec73a01317f4be9898327f885257ecaabe7658e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-y, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, CEPH_POINT_RELEASE=) 2024-02-07T13:35:33.964 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:33 smithi094 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.y.service: Deactivated successfully. 2024-02-07T13:35:33.964 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:33 smithi094 systemd[1]: Stopped Ceph mgr.y for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:35:33.965 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Feb 07 13:35:33 smithi094 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.y.service: Consumed 52.808s CPU time. 2024-02-07T13:35:33.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:35:33.978 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-07T13:35:33.978 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-07T13:35:33.978 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.x 2024-02-07T13:35:34.343 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:35:34 smithi138 systemd[1]: Stopping Ceph mgr.x for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:36.093 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:35:35 smithi138 podman[43390]: 2024-02-07 13:35:35.735559574 +0000 UTC m=+1.566812477 container died dddb105e80c14d15a74676f4c52b1592ccdc753a589a61a9debce5358e663662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux ) 2024-02-07T13:35:38.575 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:35:38 smithi138 podman[43390]: 2024-02-07 13:35:38.333524341 +0000 UTC m=+4.164777232 container cleanup dddb105e80c14d15a74676f4c52b1592ccdc753a589a61a9debce5358e663662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:35:38.576 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Feb 07 13:35:38 smithi138 bash[43390]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-x 2024-02-07T13:35:38.681 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.x.service' 2024-02-07T13:35:38.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:35:38.716 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-07T13:35:38.717 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-07T13:35:38.717 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.z 2024-02-07T13:35:39.030 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:35:38 smithi190 systemd[1]: Stopping Ceph mgr.z for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:39.617 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:35:39 smithi190 podman[43819]: 2024-02-07 13:35:39.35934075 +0000 UTC m=+0.453785164 container died 4df78f4d0932eaffeed1fca88eaeffc93740d0b661cad497ff6f9b380a657c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, io.buildah.version=1.33.2) 2024-02-07T13:35:39.876 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:35:39 smithi190 podman[43819]: 2024-02-07 13:35:39.692175794 +0000 UTC m=+0.786620207 container cleanup 4df78f4d0932eaffeed1fca88eaeffc93740d0b661cad497ff6f9b380a657c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-07T13:35:39.876 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:35:39 smithi190 bash[43819]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z 2024-02-07T13:35:40.103 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.z.service' 2024-02-07T13:35:40.128 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:35:39 smithi190 podman[43838]: 2024-02-07 13:35:39.876791693 +0000 UTC m=+0.518702769 container remove 4df78f4d0932eaffeed1fca88eaeffc93740d0b661cad497ff6f9b380a657c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T13:35:40.128 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:35:39 smithi190 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-02-07T13:35:40.128 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:35:40 smithi190 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-02-07T13:35:40.128 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:35:40 smithi190 systemd[1]: Stopped Ceph mgr.z for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:35:40.128 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Feb 07 13:35:40 smithi190 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@mgr.z.service: Consumed 16.375s CPU time. 2024-02-07T13:35:40.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:35:40.139 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-07T13:35:40.139 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-07T13:35:40.139 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.0 2024-02-07T13:35:40.436 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:40 smithi094 systemd[1]: Stopping Ceph osd.0 for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:40.763 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:40 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0[43272]: 2024-02-07T13:35:40.436+0000 7f1f35836700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:35:40.763 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:40 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0[43272]: 2024-02-07T13:35:40.436+0000 7f1f35836700 -1 osd.0 31 *** Got signal Terminated *** 2024-02-07T13:35:40.763 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:40 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0[43272]: 2024-02-07T13:35:40.436+0000 7f1f35836700 -1 osd.0 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-07T13:35:46.151 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:45 smithi094 podman[59102]: 2024-02-07 13:35:45.854354535 +0000 UTC m=+5.524636999 container died d38848e5502443e869299e70df733ffcb5b474effe1177590d13bc2618cbf846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-07T13:35:46.513 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:46 smithi094 podman[59102]: 2024-02-07 13:35:46.33430183 +0000 UTC m=+6.004584301 container cleanup d38848e5502443e869299e70df733ffcb5b474effe1177590d13bc2618cbf846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-07T13:35:46.513 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:46 smithi094 bash[59102]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0 2024-02-07T13:35:46.946 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:46 smithi094 podman[59126]: 2024-02-07 13:35:46.602614649 +0000 UTC m=+0.744638056 container remove d38848e5502443e869299e70df733ffcb5b474effe1177590d13bc2618cbf846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-07T13:35:47.263 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:46 smithi094 podman[59224]: 2024-02-07 13:35:46.849562863 +0000 UTC m=+0.017662176 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:35:47.263 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:46 smithi094 podman[59224]: 2024-02-07 13:35:46.961756981 +0000 UTC m=+0.129856281 container create 57b141b2b80e3761348f3e84b12ecac3e87efc9363f5994ae8661041d596b9cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T13:35:47.263 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:47 smithi094 podman[59224]: 2024-02-07 13:35:47.172880455 +0000 UTC m=+0.340979756 container init 57b141b2b80e3761348f3e84b12ecac3e87efc9363f5994ae8661041d596b9cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2) 2024-02-07T13:35:47.263 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:47 smithi094 podman[59224]: 2024-02-07 13:35:47.178162305 +0000 UTC m=+0.346261611 container start 57b141b2b80e3761348f3e84b12ecac3e87efc9363f5994ae8661041d596b9cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-07T13:35:47.763 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:47 smithi094 podman[59224]: 2024-02-07 13:35:47.264430404 +0000 UTC m=+0.432529712 container attach 57b141b2b80e3761348f3e84b12ecac3e87efc9363f5994ae8661041d596b9cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True) 2024-02-07T13:35:47.763 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 07 13:35:47 smithi094 podman[59224]: 2024-02-07 13:35:47.469893509 +0000 UTC m=+0.637992824 container died 57b141b2b80e3761348f3e84b12ecac3e87efc9363f5994ae8661041d596b9cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-07T13:35:48.208 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.0.service' 2024-02-07T13:35:48.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:35:48.247 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-07T13:35:48.248 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-07T13:35:48.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.1 2024-02-07T13:35:48.547 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:48 smithi138 systemd[1]: Stopping Ceph osd.1 for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:48.843 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:48 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1[35112]: 2024-02-07T13:35:48.545+0000 7fa28d4b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:35:48.843 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:48 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1[35112]: 2024-02-07T13:35:48.545+0000 7fa28d4b4700 -1 osd.1 31 *** Got signal Terminated *** 2024-02-07T13:35:48.843 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:48 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1[35112]: 2024-02-07T13:35:48.545+0000 7fa28d4b4700 -1 osd.1 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-07T13:35:54.262 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:53 smithi138 podman[43525]: 2024-02-07 13:35:53.985687997 +0000 UTC m=+5.552070410 container died 70949210762fb0b7d1f2bde85e5554574bba6c14e80536df03d5c6665fa258ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, io.buildah.version=1.33.2, ceph=True) 2024-02-07T13:35:54.593 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:54 smithi138 podman[43525]: 2024-02-07 13:35:54.369139228 +0000 UTC m=+5.935521644 container cleanup 70949210762fb0b7d1f2bde85e5554574bba6c14e80536df03d5c6665fa258ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux ) 2024-02-07T13:35:54.593 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:54 smithi138 bash[43525]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1 2024-02-07T13:35:54.940 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:54 smithi138 podman[43559]: 2024-02-07 13:35:54.620027435 +0000 UTC m=+0.628187481 container remove 70949210762fb0b7d1f2bde85e5554574bba6c14e80536df03d5c6665fa258ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131) 2024-02-07T13:35:55.247 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:54 smithi138 podman[43655]: 2024-02-07 13:35:54.84404639 +0000 UTC m=+0.017721565 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:35:55.247 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:54 smithi138 podman[43655]: 2024-02-07 13:35:54.971055014 +0000 UTC m=+0.144730231 container create 87bed626fef767b7858f2e441bad9aea9328b8328ed51c5856693e3811fceb5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-07T13:35:55.247 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:55 smithi138 podman[43655]: 2024-02-07 13:35:55.164934733 +0000 UTC m=+0.338609929 container init 87bed626fef767b7858f2e441bad9aea9328b8328ed51c5856693e3811fceb5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-07T13:35:55.248 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:55 smithi138 podman[43655]: 2024-02-07 13:35:55.170729851 +0000 UTC m=+0.344405026 container start 87bed626fef767b7858f2e441bad9aea9328b8328ed51c5856693e3811fceb5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.build-date=20240131) 2024-02-07T13:35:55.248 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:55 smithi138 podman[43655]: 2024-02-07 13:35:55.24790045 +0000 UTC m=+0.421575640 container attach 87bed626fef767b7858f2e441bad9aea9328b8328ed51c5856693e3811fceb5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2) 2024-02-07T13:35:55.593 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:55 smithi138 podman[43655]: 2024-02-07 13:35:55.455465403 +0000 UTC m=+0.629140593 container died 87bed626fef767b7858f2e441bad9aea9328b8328ed51c5856693e3811fceb5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-07T13:35:55.997 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.1.service' 2024-02-07T13:35:56.023 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:55 smithi138 podman[43701]: 2024-02-07 13:35:55.97586977 +0000 UTC m=+0.515198470 container remove 87bed626fef767b7858f2e441bad9aea9328b8328ed51c5856693e3811fceb5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:35:56.023 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:55 smithi138 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.1.service: Deactivated successfully. 2024-02-07T13:35:56.023 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:55 smithi138 systemd[1]: Stopped Ceph osd.1 for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:35:56.024 INFO:journalctl@ceph.osd.1.smithi138.stdout:Feb 07 13:35:55 smithi138 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.1.service: Consumed 4.204s CPU time. 2024-02-07T13:35:56.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:35:56.039 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-07T13:35:56.039 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-07T13:35:56.039 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.2 2024-02-07T13:35:56.339 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:35:56 smithi190 systemd[1]: Stopping Ceph osd.2 for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:35:56.780 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:35:56 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2[35992]: 2024-02-07T13:35:56.338+0000 7fdbbf372700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:35:56.780 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:35:56 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2[35992]: 2024-02-07T13:35:56.338+0000 7fdbbf372700 -1 osd.2 31 *** Got signal Terminated *** 2024-02-07T13:35:56.780 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:35:56 smithi190 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2[35992]: 2024-02-07T13:35:56.338+0000 7fdbbf372700 -1 osd.2 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-07T13:36:02.078 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:01 smithi190 podman[43964]: 2024-02-07 13:36:01.804307381 +0000 UTC m=+5.565270664 container died 5631c5cf555947d18b073e0745b32a905df9cfe524afb67a3dea8855e68c083c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-07T13:36:02.402 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:02 smithi190 podman[43964]: 2024-02-07 13:36:02.176989083 +0000 UTC m=+5.937952367 container cleanup 5631c5cf555947d18b073e0745b32a905df9cfe524afb67a3dea8855e68c083c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:36:02.402 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:02 smithi190 bash[43964]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2 2024-02-07T13:36:02.717 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:02 smithi190 podman[43987]: 2024-02-07 13:36:02.402930919 +0000 UTC m=+0.594454187 container remove 5631c5cf555947d18b073e0745b32a905df9cfe524afb67a3dea8855e68c083c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, ceph=True, org.label-schema.vendor=CentOS) 2024-02-07T13:36:03.013 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:02 smithi190 podman[44083]: 2024-02-07 13:36:02.621146788 +0000 UTC m=+0.018653395 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:36:03.013 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:02 smithi190 podman[44083]: 2024-02-07 13:36:02.720242053 +0000 UTC m=+0.117748648 container create f9bd5400618e33f0b90fd43b8181e16ceaae9486565859a96d0bada53c472d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2) 2024-02-07T13:36:03.014 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:02 smithi190 podman[44083]: 2024-02-07 13:36:02.930853603 +0000 UTC m=+0.328360200 container init f9bd5400618e33f0b90fd43b8181e16ceaae9486565859a96d0bada53c472d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131) 2024-02-07T13:36:03.014 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:02 smithi190 podman[44083]: 2024-02-07 13:36:02.936523716 +0000 UTC m=+0.334030320 container start f9bd5400618e33f0b90fd43b8181e16ceaae9486565859a96d0bada53c472d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-07T13:36:03.014 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:03 smithi190 podman[44083]: 2024-02-07 13:36:03.014282722 +0000 UTC m=+0.411789323 container attach f9bd5400618e33f0b90fd43b8181e16ceaae9486565859a96d0bada53c472d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:36:03.280 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:03 smithi190 podman[44083]: 2024-02-07 13:36:03.220783618 +0000 UTC m=+0.618290226 container died f9bd5400618e33f0b90fd43b8181e16ceaae9486565859a96d0bada53c472d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-07T13:36:03.760 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.2.service' 2024-02-07T13:36:03.780 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:03 smithi190 podman[44129]: 2024-02-07 13:36:03.734037856 +0000 UTC m=+0.508620580 container remove f9bd5400618e33f0b90fd43b8181e16ceaae9486565859a96d0bada53c472d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-osd-2-deactivate, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-07T13:36:03.780 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:03 smithi190 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.2.service: Deactivated successfully. 2024-02-07T13:36:03.780 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:03 smithi190 systemd[1]: Stopped Ceph osd.2 for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:36:03.780 INFO:journalctl@ceph.osd.2.smithi190.stdout:Feb 07 13:36:03 smithi190 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@osd.2.service: Consumed 3.668s CPU time. 2024-02-07T13:36:03.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:36:03.799 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-07T13:36:03.799 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-07T13:36:03.799 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@rgw.foo.a 2024-02-07T13:36:04.092 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:36:03 smithi094 systemd[1]: Stopping Ceph rgw.foo.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:36:04.513 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:36:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-rgw-foo-a[49313]: 2024-02-07T13:36:04.122+0000 7f8c9cb8e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:36:04.513 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:36:04 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-rgw-foo-a[49313]: 2024-02-07T13:36:04.122+0000 7f8cab3f9ac0 -1 shutting down 2024-02-07T13:36:14.465 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:36:14 smithi094 bash[59354]: time="2024-02-07T13:36:14Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-07T13:36:14.465 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:36:14 smithi094 podman[59354]: 2024-02-07 13:36:14.14996432 +0000 UTC m=+10.147104204 container died b518cef04ad6a897b8ec1859337cf27a64be62a02f327e4b7ff04fe9059badab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131) 2024-02-07T13:36:14.763 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:36:14 smithi094 podman[59354]: 2024-02-07 13:36:14.622704146 +0000 UTC m=+10.619844067 container cleanup b518cef04ad6a897b8ec1859337cf27a64be62a02f327e4b7ff04fe9059badab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-rgw-foo-a, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:36:14.763 INFO:journalctl@ceph.rgw.foo.a.smithi094.stdout:Feb 07 13:36:14 smithi094 bash[59354]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-rgw-foo-a 2024-02-07T13:36:15.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@rgw.foo.a.service' 2024-02-07T13:36:15.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:36:15.086 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-07T13:36:15.086 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-07T13:36:15.086 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@prometheus.a 2024-02-07T13:36:15.378 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 systemd[1]: Stopping Ceph prometheus.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:36:15.659 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.378Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a[40940]: ts=2024-02-07T13:36:15.379Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-07T13:36:15.660 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 podman[43786]: 2024-02-07 13:36:15.389149277 +0000 UTC m=+0.112876125 container died 98a954e2991d46265567f51ed6a1bf465b67ccfcb1828a194f4ebf7d8e7792dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:36:16.009 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 podman[43786]: 2024-02-07 13:36:15.750559384 +0000 UTC m=+0.474286239 container cleanup 98a954e2991d46265567f51ed6a1bf465b67ccfcb1828a194f4ebf7d8e7792dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:36:16.009 INFO:journalctl@ceph.prometheus.a.smithi138.stdout:Feb 07 13:36:15 smithi138 bash[43786]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-prometheus-a 2024-02-07T13:36:16.165 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@prometheus.a.service' 2024-02-07T13:36:16.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:36:16.201 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-07T13:36:16.201 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-07T13:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:36:17.455 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:36:17 smithi094 systemd[1]: Stopping Ceph alertmanager.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:36:17.720 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:36:17 smithi094 ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a[56705]: ts=2024-02-07T13:36:17.455Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-07T13:36:17.720 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:36:17 smithi094 podman[59712]: 2024-02-07 13:36:17.476008699 +0000 UTC m=+0.121199095 container died 4012c0c6d40faacb8c1035ebb1f092a176d8ad8ab3b9ba114dcbd64f57d9273e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:36:18.013 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:36:17 smithi094 podman[59712]: 2024-02-07 13:36:17.895266812 +0000 UTC m=+0.540457247 container cleanup 4012c0c6d40faacb8c1035ebb1f092a176d8ad8ab3b9ba114dcbd64f57d9273e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:36:18.013 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:36:17 smithi094 bash[59712]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a 2024-02-07T13:36:18.338 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:36:18 smithi094 podman[59731]: 2024-02-07 13:36:18.046488228 +0000 UTC m=+0.569839004 container remove 4012c0c6d40faacb8c1035ebb1f092a176d8ad8ab3b9ba114dcbd64f57d9273e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:36:18.338 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:36:18 smithi094 podman[59731]: 2024-02-07 13:36:18.122218346 +0000 UTC m=+0.645569117 volume remove ee4eb65456d9923e35a4dae8e0e1232cea0f51909f297612b71ed1668bd0990f 2024-02-07T13:36:18.763 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:36:18 smithi094 systemd[1]: Stopping Ceph node-exporter.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:36:18.764 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:36:18 smithi094 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-07T13:36:18.764 INFO:journalctl@ceph.alertmanager.a.smithi094.stdout:Feb 07 13:36:18 smithi094 systemd[1]: Stopped Ceph alertmanager.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:36:19.108 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:36:18 smithi094 podman[59849]: 2024-02-07 13:36:18.804988627 +0000 UTC m=+0.126609655 container died 4ba4235945c53108f757cf81e0ec2087531d6f25f4d439dff87de85be34954f3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:36:19.369 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:36:19 smithi094 podman[59849]: 2024-02-07 13:36:19.350783659 +0000 UTC m=+0.672404691 container cleanup 4ba4235945c53108f757cf81e0ec2087531d6f25f4d439dff87de85be34954f3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:36:19.369 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:36:19 smithi094 bash[59849]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a 2024-02-07T13:36:19.975 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:36:19 smithi094 podman[59868]: 2024-02-07 13:36:19.577201833 +0000 UTC m=+0.776880352 container remove 4ba4235945c53108f757cf81e0ec2087531d6f25f4d439dff87de85be34954f3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:36:19.975 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:36:19 smithi094 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-07T13:36:19.975 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:36:19 smithi094 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-07T13:36:19.976 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:36:19 smithi094 systemd[1]: Stopped Ceph node-exporter.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:36:19.976 INFO:journalctl@ceph.node-exporter.a.smithi094.stdout:Feb 07 13:36:19 smithi094 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@node-exporter.a.service: Consumed 1.786s CPU time. 2024-02-07T13:36:20.326 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-07T13:36:20.514 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:36:21.593 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:36:21 smithi138 systemd[1]: Stopping Ceph grafana.a for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:36:21.871 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:36:21 smithi138 podman[44122]: 2024-02-07 13:36:21.617208497 +0000 UTC m=+0.116383107 container died a3e0fd96e69264a3b02d707b1a532aa33003748ab27ed7bd1aa649b92f40643c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., maintainer=Paul Cuzner , version=8.5, io.openshift.expose-services=, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, distribution-scope=public, io.buildah.version=1.32.0, release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-02-07T13:36:22.171 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:36:21 smithi138 podman[44122]: 2024-02-07 13:36:21.953511337 +0000 UTC m=+0.452685938 container cleanup a3e0fd96e69264a3b02d707b1a532aa33003748ab27ed7bd1aa649b92f40643c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, version=8.5, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, distribution-scope=public, release=239.1651231664, name=ubi8, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, description=Ceph Grafana Container, vcs-type=git, architecture=x86_64, maintainer=Paul Cuzner ) 2024-02-07T13:36:22.171 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:36:21 smithi138 bash[44122]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a 2024-02-07T13:36:22.427 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:36:22 smithi138 podman[44141]: 2024-02-07 13:36:22.171571147 +0000 UTC m=+0.557024900 container remove a3e0fd96e69264a3b02d707b1a532aa33003748ab27ed7bd1aa649b92f40643c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, io.buildah.version=1.32.0, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, vendor=Red Hat, Inc., release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, distribution-scope=public, maintainer=Paul Cuzner , version=8.5) 2024-02-07T13:36:22.427 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:36:22 smithi138 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-07T13:36:22.427 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:36:22 smithi138 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-07T13:36:22.427 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:36:22 smithi138 systemd[1]: Stopped Ceph grafana.a for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:36:22.427 INFO:journalctl@ceph.grafana.a.smithi138.stdout:Feb 07 13:36:22 smithi138 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@grafana.a.service: Consumed 2.709s CPU time. 2024-02-07T13:36:22.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:36:22 smithi138 systemd[1]: Stopping Ceph node-exporter.b for edf33ada-c5bc-11ee-95b6-87774f69a715... 2024-02-07T13:36:23.166 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:36:22 smithi138 podman[44260]: 2024-02-07 13:36:22.903981503 +0000 UTC m=+0.123405711 container died eb85e60947b4d4e9495b762cba3db0b79d46b8d523c8723b6152ef961ab389c3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:36:23.524 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:36:23 smithi138 podman[44260]: 2024-02-07 13:36:23.265293356 +0000 UTC m=+0.484717563 container cleanup eb85e60947b4d4e9495b762cba3db0b79d46b8d523c8723b6152ef961ab389c3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:36:23.524 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:36:23 smithi138 bash[44260]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b 2024-02-07T13:36:23.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:36:23 smithi138 podman[44282]: 2024-02-07 13:36:23.524548161 +0000 UTC m=+0.625569783 container remove eb85e60947b4d4e9495b762cba3db0b79d46b8d523c8723b6152ef961ab389c3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-edf33ada-c5bc-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:36:23.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:36:23 smithi138 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-07T13:36:23.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:36:23 smithi138 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-07T13:36:23.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:36:23 smithi138 systemd[1]: Stopped Ceph node-exporter.b for edf33ada-c5bc-11ee-95b6-87774f69a715. 2024-02-07T13:36:23.843 INFO:journalctl@ceph.node-exporter.b.smithi138.stdout:Feb 07 13:36:23 smithi138 systemd[1]: ceph-edf33ada-c5bc-11ee-95b6-87774f69a715@node-exporter.b.service: Consumed 1.787s CPU time. 2024-02-07T13:36:24.381 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-07T13:36:24.572 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:36:25.670 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:36:25.704 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:36:25.739 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:36:25.767 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-07T13:36:25.769 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/remote/smithi094/crash 2024-02-07T13:36:25.770 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/crash -- . 2024-02-07T13:36:25.808 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-07T13:36:25.808 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-02-07T13:36:25.810 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/remote/smithi138/crash 2024-02-07T13:36:25.811 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/crash -- . 2024-02-07T13:36:25.853 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-07T13:36:25.853 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-02-07T13:36:25.856 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/remote/smithi190/crash 2024-02-07T13:36:25.857 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/crash -- . 2024-02-07T13:36:25.895 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-07T13:36:25.895 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2024-02-07T13:36:25.896 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-07T13:36:25.896 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T13:36:25.929 INFO:teuthology.orchestra.run.smithi094.stdout:2024-02-07T13:32:31.770541+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T13:36:25.929 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-07T13:36:25.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[SEC\]' /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T13:36:26.000 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]' /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T13:36:26.071 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[WRN\]' /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T13:36:26.140 INFO:teuthology.orchestra.run.smithi094.stdout:2024-02-07T13:32:31.770541+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-07T13:36:26.140 INFO:tasks.cephadm:Compressing logs... 2024-02-07T13:36:26.140 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-07T13:36:26.182 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-07T13:36:26.185 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-07T13:36:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:: No such file or directory 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mon.a.log 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mgr.y.log 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-osd.0.log 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log: /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mgr.y.log: 57.7% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr: 89.4% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-07T13:36:26.210 INFO:teuthology.orchestra.run.smithi094.stderr: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-07T13:36:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log: 82.6% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-07T13:36:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-volume.log: 84.9% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-07T13:36:26.213 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-07T13:36:26.213 INFO:teuthology.orchestra.run.smithi190.stderr:: No such file or directory 2024-02-07T13:36:26.213 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-07T13:36:26.213 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mgr.z.log 2024-02-07T13:36:26.213 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-07T13:36:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.audit.log: /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log 2024-02-07T13:36:26.214 INFO:teuthology.orchestra.run.smithi190.stderr: 90.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-07T13:36:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-osd.2.log 2024-02-07T13:36:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.cephadm.log: /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-07T13:36:26.214 INFO:teuthology.orchestra.run.smithi190.stderr: 81.2% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-07T13:36:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mon.c.log 2024-02-07T13:36:26.214 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-07T13:36:26.214 INFO:teuthology.orchestra.run.smithi190.stderr: 84.4% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-07T13:36:26.214 INFO:teuthology.orchestra.run.smithi094.stderr: 90.5% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-07T13:36:26.217 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-07T13:36:26.217 INFO:teuthology.orchestra.run.smithi138.stderr:: No such file or directory 2024-02-07T13:36:26.217 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-07T13:36:26.217 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-osd.1.log 2024-02-07T13:36:26.217 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mgr.x.log 2024-02-07T13:36:26.217 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-07T13:36:26.217 INFO:teuthology.orchestra.run.smithi138.stderr:gzip/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mgr.x.log: -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log 2024-02-07T13:36:26.217 INFO:teuthology.orchestra.run.smithi138.stderr: 89.4% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-07T13:36:26.217 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-07T13:36:26.218 INFO:teuthology.orchestra.run.smithi138.stderr: 81.2% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-07T13:36:26.218 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mon.b.log 2024-02-07T13:36:26.218 INFO:teuthology.orchestra.run.smithi138.stderr: 81.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-07T13:36:26.218 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-volume.log: 84.2% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-07T13:36:26.218 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-volume.log: /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mon.c.log: 90.3% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mgr.z.log.gz 2024-02-07T13:36:26.218 INFO:teuthology.orchestra.run.smithi190.stderr: 90.5% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-07T13:36:26.219 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mon.b.log: 90.5% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-07T13:36:26.221 INFO:teuthology.orchestra.run.smithi138.stderr: 90.2% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mgr.x.log.gz 2024-02-07T13:36:26.270 INFO:teuthology.orchestra.run.smithi190.stderr: 92.3% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mon.c.log.gz 2024-02-07T13:36:26.270 INFO:teuthology.orchestra.run.smithi138.stderr: 92.3% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mon.b.log.gz 2024-02-07T13:36:26.297 INFO:teuthology.orchestra.run.smithi094.stderr: 88.9% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mgr.y.log.gz 2024-02-07T13:36:26.382 INFO:teuthology.orchestra.run.smithi094.stderr: 91.4% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-mon.a.log.gz 2024-02-07T13:36:26.472 INFO:teuthology.orchestra.run.smithi190.stderr: 94.3% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-osd.2.log.gz 2024-02-07T13:36:26.474 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-02-07T13:36:26.474 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m0.275s 2024-02-07T13:36:26.474 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m0.334s 2024-02-07T13:36:26.474 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.025s 2024-02-07T13:36:26.483 INFO:teuthology.orchestra.run.smithi094.stderr: 94.2% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-osd.0.log.gz 2024-02-07T13:36:26.485 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-02-07T13:36:26.485 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m0.291s 2024-02-07T13:36:26.485 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m0.544s 2024-02-07T13:36:26.485 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.035s 2024-02-07T13:36:26.516 INFO:teuthology.orchestra.run.smithi138.stderr: 94.2% -- replaced with /var/log/ceph/edf33ada-c5bc-11ee-95b6-87774f69a715/ceph-osd.1.log.gz 2024-02-07T13:36:26.517 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-02-07T13:36:26.518 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m0.318s 2024-02-07T13:36:26.518 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m0.365s 2024-02-07T13:36:26.518 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.040s 2024-02-07T13:36:26.518 INFO:tasks.cephadm:Archiving logs... 2024-02-07T13:36:26.518 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/remote/smithi094/log 2024-02-07T13:36:26.519 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-07T13:36:26.646 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/remote/smithi138/log 2024-02-07T13:36:26.647 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-07T13:36:26.733 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/remote/smithi190/log 2024-02-07T13:36:26.734 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-07T13:36:26.815 INFO:tasks.cephadm:Removing cluster... 2024-02-07T13:36:26.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 --force 2024-02-07T13:36:26.993 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:36:27.106 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 --force 2024-02-07T13:36:27.290 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:36:27.404 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid edf33ada-c5bc-11ee-95b6-87774f69a715 --force 2024-02-07T13:36:27.597 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: edf33ada-c5bc-11ee-95b6-87774f69a715 2024-02-07T13:36:27.711 INFO:tasks.cephadm:Removing cephadm ... 2024-02-07T13:36:27.712 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-07T13:36:27.728 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-07T13:36:27.744 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-07T13:36:27.758 INFO:tasks.cephadm:Teardown complete 2024-02-07T13:36:27.759 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-02-07T13:36:27.768 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi094:/dev/vg_nvme/lv_1... 2024-02-07T13:36:27.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo nvme disconnect -n lv_1 2024-02-07T13:36:27.917 INFO:teuthology.orchestra.run.smithi094.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-07T13:36:27.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:27.918 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi094:/dev/vg_nvme/lv_2... 2024-02-07T13:36:27.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo nvme disconnect -n lv_2 2024-02-07T13:36:28.068 INFO:teuthology.orchestra.run.smithi094.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-07T13:36:28.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:28.069 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi094:/dev/vg_nvme/lv_3... 2024-02-07T13:36:28.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo nvme disconnect -n lv_3 2024-02-07T13:36:28.257 INFO:teuthology.orchestra.run.smithi094.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-07T13:36:28.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:28.258 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi094:/dev/vg_nvme/lv_4... 2024-02-07T13:36:28.258 DEBUG:teuthology.orchestra.run.smithi094:> sudo nvme disconnect -n lv_4 2024-02-07T13:36:28.393 INFO:teuthology.orchestra.run.smithi094.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-07T13:36:28.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:28.395 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-07T13:36:28.395 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/scratch_devs 2024-02-07T13:36:28.422 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi138:/dev/vg_nvme/lv_1... 2024-02-07T13:36:28.422 DEBUG:teuthology.orchestra.run.smithi138:> sudo nvme disconnect -n lv_1 2024-02-07T13:36:28.572 INFO:teuthology.orchestra.run.smithi138.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-07T13:36:28.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:28.574 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi138:/dev/vg_nvme/lv_2... 2024-02-07T13:36:28.574 DEBUG:teuthology.orchestra.run.smithi138:> sudo nvme disconnect -n lv_2 2024-02-07T13:36:28.730 INFO:teuthology.orchestra.run.smithi138.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-07T13:36:28.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:28.732 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi138:/dev/vg_nvme/lv_3... 2024-02-07T13:36:28.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo nvme disconnect -n lv_3 2024-02-07T13:36:28.871 INFO:teuthology.orchestra.run.smithi138.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-07T13:36:28.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:28.872 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi138:/dev/vg_nvme/lv_4... 2024-02-07T13:36:28.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo nvme disconnect -n lv_4 2024-02-07T13:36:29.037 INFO:teuthology.orchestra.run.smithi138.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-07T13:36:29.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:29.039 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-02-07T13:36:29.039 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/scratch_devs 2024-02-07T13:36:29.069 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi190:/dev/vg_nvme/lv_1... 2024-02-07T13:36:29.069 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme disconnect -n lv_1 2024-02-07T13:36:29.213 INFO:teuthology.orchestra.run.smithi190.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-07T13:36:29.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:29.215 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi190:/dev/vg_nvme/lv_2... 2024-02-07T13:36:29.215 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme disconnect -n lv_2 2024-02-07T13:36:29.361 INFO:teuthology.orchestra.run.smithi190.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-07T13:36:29.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:29.363 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi190:/dev/vg_nvme/lv_3... 2024-02-07T13:36:29.363 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme disconnect -n lv_3 2024-02-07T13:36:29.511 INFO:teuthology.orchestra.run.smithi190.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-07T13:36:29.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:29.513 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi190:/dev/vg_nvme/lv_4... 2024-02-07T13:36:29.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme disconnect -n lv_4 2024-02-07T13:36:29.654 INFO:teuthology.orchestra.run.smithi190.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-07T13:36:29.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:29.656 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-02-07T13:36:29.656 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/scratch_devs 2024-02-07T13:36:29.686 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-07T13:36:29.696 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-07T13:36:29.696 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:36:29.699 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:36:29.701 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:36:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:36:29.715 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:36:29.716 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:36:29.716 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-02-07T13:36:29.716 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 -711us[ -711us] +/- 91ms 2024-02-07T13:36:29.716 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 +470us[ +476us] +/- 55ms 2024-02-07T13:36:29.716 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -62us[ -56us] +/- 73ms 2024-02-07T13:36:29.716 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:36:29.720 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:36:29.720 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-02-07T13:36:29.720 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 -698us[ -698us] +/- 91ms 2024-02-07T13:36:29.720 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 +483us[ +492us] +/- 55ms 2024-02-07T13:36:29.720 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -57us[ -48us] +/- 73ms 2024-02-07T13:36:29.720 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:36:29.741 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:36:29.747 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:36:29.747 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-02-07T13:36:29.747 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 -701us[ -701us] +/- 91ms 2024-02-07T13:36:29.747 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 +479us[ +481us] +/- 55ms 2024-02-07T13:36:29.747 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -66us[ -66us] +/- 73ms 2024-02-07T13:36:29.747 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:36:29.747 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-07T13:36:29.759 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-07T13:36:29.760 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-07T13:36:29.773 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-07T13:36:29.849 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-07T13:36:29.923 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-07T13:36:29.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:36:30.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:30.030 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:36:30.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:30.061 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:36:30.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:30.092 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-07T13:36:30.101 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-07T13:36:30.113 INFO:teuthology.task.internal:Duration was 673.895642 seconds 2024-02-07T13:36:30.113 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-07T13:36:30.127 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-07T13:36:30.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-07T13:36:30.130 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-07T13:36:30.133 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-07T13:36:30.172 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:36:30.175 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:36:30.176 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:36:30.631 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-07T13:36:30.631 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-02-07T13:36:30.631 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-07T13:36:30.651 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-02-07T13:36:30.651 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-07T13:36:30.677 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-02-07T13:36:30.678 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-07T13:36:30.700 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-07T13:36:30.701 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-07T13:36:30.703 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-07T13:36:30.720 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-07T13:36:30.770 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-07T13:36:30.770 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-07T13:36:30.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-07T13:36:30.790 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-07T13:36:30.963 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-07T13:36:30.973 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-07T13:36:30.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-07T13:36:31.000 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-07T13:36:31.026 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-07T13:36:31.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-07T13:36:31.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:36:31.066 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:36:31.069 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:36:31.089 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-02-07T13:36:31.092 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-02-07T13:36:31.119 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-02-07T13:36:31.143 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:36:31.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:31.177 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:36:31.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:31.191 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:36:31.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:36:31.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-07T13:36:31.215 INFO:teuthology.task.internal:Transferring archived files... 2024-02-07T13:36:31.216 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/remote/smithi094 2024-02-07T13:36:31.216 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-07T13:36:31.255 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/remote/smithi138 2024-02-07T13:36:31.256 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-07T13:36:31.291 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802/remote/smithi190 2024-02-07T13:36:31.291 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-07T13:36:31.326 INFO:teuthology.task.internal:Removing archive directory... 2024-02-07T13:36:31.327 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-07T13:36:31.329 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-07T13:36:31.331 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-07T13:36:31.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-07T13:36:31.388 INFO:teuthology.task.internal:Not uploading archives. 2024-02-07T13:36:31.388 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-07T13:36:31.400 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-07T13:36:31.401 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-07T13:36:31.403 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-07T13:36:31.405 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-07T13:36:31.415 INFO:teuthology.orchestra.run.smithi094.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 7 13:36 /home/ubuntu/cephtest 2024-02-07T13:36:31.417 INFO:teuthology.orchestra.run.smithi138.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 7 13:36 /home/ubuntu/cephtest 2024-02-07T13:36:31.432 INFO:teuthology.orchestra.run.smithi190.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 7 13:36 /home/ubuntu/cephtest 2024-02-07T13:36:31.433 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-07T13:36:31.443 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-07T13:36:31.467 INFO:teuthology.nuke:Checking targets against current locks 2024-02-07T13:36:31.493 DEBUG:teuthology.nuke:shortname: smithi094 2024-02-07T13:36:31.493 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T13:36:31.517 DEBUG:teuthology.nuke:shortname: smithi138 2024-02-07T13:36:31.517 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T13:36:31.522 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 13:16:37.680348', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOPf41Y98XLj3FwHreyOAexv4hXn/EvTZsS0oWIJ9YL92vybX289iPq9bLdloH0rpnzDapH6/ieA0Hhn1/IoN1g='} 2024-02-07T13:36:31.540 DEBUG:teuthology.nuke:shortname: smithi190 2024-02-07T13:36:31.540 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T13:36:31.545 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 13:16:37.684149', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEhqXTkUlRo6RvIzmZkkeo1HcOpjPWPnEK9mI4G6XkuRdHTvqUb1J5PIkgNhgSgkHPkBaawlgr+uVZfuk6jDcFc='} 2024-02-07T13:36:31.575 INFO:teuthology.orchestra.console.smithi094:Power off 2024-02-07T13:36:31.575 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-07T13:36:31.583 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548802', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 13:16:37.682431', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFfJfD06HpTnT9dw9pbSErpDqkWD+hJYPnsZ/Nzpnjqj7C75iZPiuaql6PyO06rMwF3H1wAhImKbR7ujc00Jdrc='} 2024-02-07T13:36:31.598 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2024-02-07T13:36:31.598 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:36:31.617 INFO:teuthology.orchestra.console.smithi138:Power off 2024-02-07T13:36:31.617 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-07T13:36:31.626 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-02-07T13:36:31.639 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2024-02-07T13:36:31.639 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:36:31.653 INFO:teuthology.orchestra.console.smithi190:Power off 2024-02-07T13:36:31.654 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-07T13:36:31.662 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-02-07T13:36:31.678 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2024-02-07T13:36:31.678 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:36:31.701 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-02-07T13:36:35.628 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:36:35.663 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:36:35.702 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:36:35.739 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-02-07T13:36:35.773 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-02-07T13:36:35.812 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-02-07T13:36:39.744 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:36:39.774 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:36:39.813 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:36:39.853 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-02-07T13:36:39.883 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-02-07T13:36:39.923 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2024-02-07T13:36:39.954 INFO:teuthology.orchestra.console.smithi094:Power off completed 2024-02-07T13:36:39.984 INFO:teuthology.orchestra.console.smithi138:Power off completed 2024-02-07T13:36:40.024 INFO:teuthology.orchestra.console.smithi190:Power off completed 2024-02-07T13:36:40.128 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} duration: 673.8956422805786 failure_reason: '"2024-02-07T13:32:31.770541+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' owner: scheduled_gabrioux@teuthology success: false 2024-02-07T13:36:40.128 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-07T13:36:40.212 INFO:teuthology.run:FAIL